| Age | Commit message (Collapse) | Author |
|---|
|
With the growing number of Intel crypto drivers, it makes sense to
group them all into a single drivers/crypto/intel/ directory.
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
pci_enable_pcie_error_reporting() enables the device to send ERR_*
Messages. Since f26e58bf6f54 ("PCI/AER: Enable error reporting when AER is
native"), the PCI core does this for all devices during enumeration, so the
driver doesn't need to do it itself.
Remove the redundant pci_enable_pcie_error_reporting() call from the
driver. Also remove the corresponding pci_disable_pcie_error_reporting()
from the driver .remove() path.
Note that this only controls ERR_* Messages from the device. An ERR_*
Message may cause the Root Port to generate an interrupt, depending on the
AER Root Error Command register managed by the AER service driver.
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Cc: qat-linux@intel.com
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The thread to arbiter mapping for 4xxx devices does not allow to
achieve optimal performance for the compression service as it makes
all the engines to compete for the same resources.
Update the logic so that a custom optimal mapping is used for the
compression service.
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
QAT_402xx is a derivative of 4xxx. Add support for that device in the
qat_4xxx driver by including the DIDs (both PF and VF), extending the
probe and the firmware loader.
402xx uses different firmware images than 4xxx. To allow that the logic
that selects the firmware images was modified.
Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The state machine functions adf_dev_init(), adf_dev_start(),
adf_dev_stop() adf_dev_shutdown() and adf_dev_shutdown_cache_cfg()
are only used internally within adf_init.c.
Do not export these functions and make them static as state transitions
are now performed using the safe function adf_dev_up() and
adf_dev_down().
This commit does not implement any functional change.
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Refactor the restart logic by moving it into the function
adf_dev_restart() which uses the safe function adf_dev_up() and
adf_dev_down().
This commit does not implement any functional change.
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The device state machine functions are unsafe and interdependent on each
other. To perform a state transition, these shall be called in a
specific order:
* device up: adf_dev_init() -> adf_dev_start()
* device down: adf_dev_stop() -> adf_dev_shutdown()
Replace all the state machine functions used in the QAT driver with the
safe wrappers adf_dev_up() and adf_dev_down().
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The sysfs `state` attribute is not protected against race conditions.
If multiple processes perform a device state transition on the same
device in parallel, unexpected behaviors might occur.
For transitioning the device state, adf_sysfs.c calls the functions
adf_dev_init(), adf_dev_start(), adf_dev_stop() and adf_dev_shutdown()
which are unprotected and interdependent on each other. To perform a
state transition, these functions needs to be called in a specific
order:
* device up: adf_dev_init() -> adf_dev_start()
* device down: adf_dev_stop() -> adf_dev_shutdown()
This change introduces the functions adf_dev_up() and adf_dev_down()
which wrap the state machine functions and protect them with a
per-device lock. These are then used in adf_sysfs.c instead of the
individual state transition functions.
Fixes: 5ee52118ac14 ("crypto: qat - expose device state through sysfs for 4xxx")
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The function adf_sysfs_init() is used by qat_4xxx to create sysfs
attributes. This is called by the probe function before starting a
device. With this sequence, there might be a chance that the sysfs
entries for configuration might be changed by a user while the driver
is performing a device bring-up causing unexpected behaviors.
Delay the creation of sysfs entries after adf_dev_start().
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Include crypto/algapi.h instead of linux/crypto.h in adf_ctl_drv.c
as this is using the low-level Crypto API. It just happens to work
currently because MODULE_ALIAS_CRYPTO was mistakenly added to
linux/crypto.h.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Use the request_complete helpers instead of calling the completion
function directly.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The functions qat_crypto_get_instance_node() and
qat_compression_get_instance_node() allow to get a QAT instance (ring
pair) on a device close to the node specified as input parameter.
When this is not possible, and a QAT device is available in the system,
these function return an instance on a remote node and they print a
message reporting that it is not possible to find a device on the specified
node. This is interpreted by people as an error rather than an info.
The print "Could not find a device on node" indicates that a kernel
application is running on a core in a socket that does not have a QAT
device directly attached to it and performance might suffer.
Due to the nature of the message, this can be considered as a debug
message, therefore drop the severity to debug and report it only once
to avoid flooding.
Suggested-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Tested-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
When preparing an AER-CTR request, the driver copies the key provided by
the user into a data structure that is accessible by the firmware.
If the target device is QAT GEN4, the key size is rounded up by 16 since
a rounded up size is expected by the device.
If the key size is rounded up before the copy, the size used for copying
the key might be bigger than the size of the region containing the key,
causing an out-of-bounds read.
Fix by doing the copy first and then update the keylen.
This is to fix the following warning reported by KASAN:
[ 138.150574] BUG: KASAN: global-out-of-bounds in qat_alg_skcipher_init_com.isra.0+0x197/0x250 [intel_qat]
[ 138.150641] Read of size 32 at addr ffffffff88c402c0 by task cryptomgr_test/2340
[ 138.150651] CPU: 15 PID: 2340 Comm: cryptomgr_test Not tainted 6.2.0-rc1+ #45
[ 138.150659] Hardware name: Intel Corporation ArcherCity/ArcherCity, BIOS EGSDCRB1.86B.0087.D13.2208261706 08/26/2022
[ 138.150663] Call Trace:
[ 138.150668] <TASK>
[ 138.150922] kasan_check_range+0x13a/0x1c0
[ 138.150931] memcpy+0x1f/0x60
[ 138.150940] qat_alg_skcipher_init_com.isra.0+0x197/0x250 [intel_qat]
[ 138.151006] qat_alg_skcipher_init_sessions+0xc1/0x240 [intel_qat]
[ 138.151073] crypto_skcipher_setkey+0x82/0x160
[ 138.151085] ? prepare_keybuf+0xa2/0xd0
[ 138.151095] test_skcipher_vec_cfg+0x2b8/0x800
Fixes: 67916c951689 ("crypto: qat - add AES-CTR support for QAT GEN4 devices")
Cc: <stable@vger.kernel.org>
Reported-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Tested-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The ZLIB format (RFC 1950) is made of deflate compressed data surrounded
by a header and a footer. The QAT accelerators support only the deflate
algorithm, therefore the header and the footer need to be inserted in
software.
This adds logic in the QAT driver to support the ZLIB format. In
particular:
* Generalize the function qat_comp_alg_compress_decompress() to allow
skipping an initial region (header) of the source and/or destination
scatter lists.
* Add logic to register the qat_zlib_deflate algorithm into the acomp
framework.
* For ZLIB compression, skip the initial portion of the destination
buffer before sending the job to the QAT accelerator and insert the
ZLIB header and footer in the callback, after the QAT request has
been processed.
* For ZLIB decompression, parse the header in the input buffer
provided by the user and verify its validity before attempting the
decompression of the buffer with QAT. Then submit the buffer to QAT
for decompression. In the callback verify the correctness of the
footer by comparing the value of the ADLER produced by QAT with the
one in the destination buffer.
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Extend qat_bl_sgl_to_bufl() to allow skipping the mapping of a region
of the source and the destination scatter lists starting from byte
zero.
This is to support the ZLIB format (RFC 1950) in the qat driver.
The ZLIB format is made of deflate compressed data surrounded by a
header and a footer. The QAT accelerators support only the deflate
algorithm, therefore the header should not be mapped since it is
inserted in software.
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Fix spelling mistakes from 'bufer' to 'buffer' in qat_common.
Also fix indentation issue caused by the spelling change.
Signed-off-by: Meadhbh Fitzpatrick <meadhbh.fitzpatrick@intel.com>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The acomp API allows to send requests with a NULL destination buffer. In
this case, the algorithm implementation needs to allocate the
destination scatter list, perform the operation and return the buffer to
the user. For decompression, data is likely to expand and be bigger than
the allocated buffer.
This implements a re-submission mechanism for decompression requests
that is triggered if the destination buffer, allocated by the driver,
is not sufficiently big to store the output from decompression.
If an overflow is detected when processing the callback for a
decompression request with a NULL destination buffer, a workqueue is
scheduled. This allocates a new scatter list of size CRYPTO_ACOMP_DST_MAX,
now 128KB, creates a new firmware scatter list and resubmits the job to
the hardware accelerator.
Suggested-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Enable deflate for QAT GEN4 devices.
This adds
(1) logic to create configuration entries at probe time for the
compression instances for QAT GEN4 devices;
(2) the implementation of QAT GEN4 specific compression operations,
required since the creation of the compression request template is
different between GEN2 and GEN4; and
(3) updates to the firmware API related to compression for GEN4.
The implementation configures the device to produce data compressed
dynamically, optimized for throughput over compression ratio.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Add infrastructure for implementing the acomp APIs in the QAT driver and
expose the deflate algorithm for QAT GEN2 devices.
This adds
(1) the compression service which includes logic to create, allocate
and handle compression instances;
(2) logic to create configuration entries at probe time for the
compression instances;
(3) updates to the firmware API for allowing the compression service;
and;
(4) a back-end for deflate that implements the acomp api for QAT GEN2
devices.
The implementation configures the device to produce data compressed
statically, optimized for throughput over compression ratio.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Rename qat_crypto_dev_config() in adf_gen2_dev_config() and relocate it
to the newly created file adf_gen2_config.c.
This function is specific to QAT GEN2 devices and will be used also to
configure the compression service.
In addition change the drivers to use the dev_config() in the hardware
data structure (which for GEN2 devices now points to
adf_gen2_dev_config()), for consistency.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Move qat_algs_alloc_flags() from qat_crypto.h to qat_bl.h as this will
be used also by the compression logic.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Move the structures qat_instance_backlog and qat_alg_req from
qat_crypto.h to qat_algs_send.h since they are not unique to crypto.
Both structures will be used by the compression service to support
requests with the CRYPTO_TFM_REQ_MAY_BACKLOG flag set.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The compression service requires an additional pre-allocated buffer for
each destination scatter list.
Extend the function qat_alg_sgl_to_bufl() to take an additional
structure that contains the dma address and the size of the extra
buffer which will be appended in the destination FW SGL.
The logic that unmaps buffers in qat_alg_free_bufl() has been changed to
start unmapping from buffer 0 instead of skipping the initial buffers
num_buff - num_mapped_bufs as that functionality was not used in the
code.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The structure qat_crypto_request_buffs which contains the source and
destination buffer lists and correspondent sizes and dma addresses is
also required for the compression service.
Rename it as qat_request_buffs and move it to qat_bl.h.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The functions qat_alg_sgl_to_bufl() and qat_alg_free_bufl() take as
argument a qat_crypto_instance and a qat_crypto_request structure.
These two structures are used only to get a reference to the
adf_accel_dev and qat_crypto_request_buffs.
In order to reuse these functions for the compression service, change
the signature so that they take adf_accel_dev and
qat_crypto_request_buffs.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Rename the functions qat_alg_sgl_to_bufl() and qat_alg_free_bufl() as
qat_bl_sgl_to_bufl() and qat_bl_free_bufl() after their relocation into
the qat_bl module.
This commit does not implement any functional change.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Move the logic that maps, unmaps and converts scatterlists into QAT
bufferlists from qat_algs.c to a new module, qat_bl.
This is to allow reuse of the logic by the data compression service.
This commit does not implement any functional change.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The value of reqsize must only be changed through the helper.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Fix to return a negative error code -EINVAL instead of 0.
Fixes: 0cec19c761e5 ("crypto: qat - add support for compression for 4xxx")
Signed-off-by: Wang Yufen <wangyufen@huawei.com>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The ADF_STATUS_PF_RUNNING bit is set after the successful initialization
of the communication between VF to PF in adf_vf2pf_notify_init().
So, it is not required to be set after the execution of the function
adf_dev_init().
Signed-off-by: Shashank Gupta <shashank.gupta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
adf_copy_key_value_data() copies data from userland to kernel, based on
a linked link provided by userland. If userland provides a circular
list (or just a very long one) then it would drive a long loop where
allocation occurs in every loop. This could lead to low memory conditions.
Adding a limit to stop endless loop.
Signed-off-by: Adam Guerin <adam.guerin@intel.com>
Co-developed-by: Ciunas Bennett <ciunas.bennett@intel.com>
Signed-off-by: Ciunas Bennett <ciunas.bennett@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
When mapping the input and output parameters, the implementations of RSA
and DH pass to the function dma_map_single() a pointer to the first
member of the structure they want to map instead of a pointer to the
actual structure.
This results in set of warnings reported by the static analyser Smatch:
drivers/crypto/qat/qat_common/qat_asym_algs.c:335 qat_dh_compute_value() error: dma_map_single_attrs() '&qat_req->in.dh.in.b' too small (8 vs 64)
drivers/crypto/qat/qat_common/qat_asym_algs.c:341 qat_dh_compute_value() error: dma_map_single_attrs() '&qat_req->out.dh.r' too small (8 vs 64)
drivers/crypto/qat/qat_common/qat_asym_algs.c:732 qat_rsa_enc() error: dma_map_single_attrs() '&qat_req->in.rsa.enc.m' too small (8 vs 64)
drivers/crypto/qat/qat_common/qat_asym_algs.c:738 qat_rsa_enc() error: dma_map_single_attrs() '&qat_req->out.rsa.enc.c' too small (8 vs 64)
drivers/crypto/qat/qat_common/qat_asym_algs.c:878 qat_rsa_dec() error: dma_map_single_attrs() '&qat_req->in.rsa.dec.c' too small (8 vs 64)
drivers/crypto/qat/qat_common/qat_asym_algs.c:884 qat_rsa_dec() error: dma_map_single_attrs() '&qat_req->out.rsa.dec.m' too small (8 vs 64)
Where the address of the first element of a structure is used as an
input for the function dma_map_single(), replace it with the address of
the structure. This fix does not introduce any functional change as the
addresses are the same.
Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
This reverts commit e48767c17718067ba21fb2ef461779ec2506f845.
In an attempt to resolve a set of warnings reported by the static
analyzer Smatch, the reverted commit improperly reduced the sizes of the
DMA mappings used for the input and output parameters for both RSA and
DH creating a mismatch (map size=8 bytes, unmap size=64 bytes).
This issue is reported when CONFIG_DMA_API_DEBUG is selected, when the
crypto self test is run. The function dma_unmap_single() reports a
warning similar to the one below, saying that the `device driver frees
DMA memory with different size`.
DMA-API: 4xxx 0000:06:00.0: device driver frees DMA memory with different size [device address=0x0000000123206c80] [map size=8 bytes] [unmap size=64 bytes]
WARNING: CPU: 0 PID: 0 at kernel/dma/debug.c:973 check_unmap+0x3d0/0x8c0\
...
Call Trace:
<IRQ>
debug_dma_unmap_page+0x5c/0x60
qat_dh_cb+0xd7/0x110 [intel_qat]
qat_alg_asym_callback+0x1a/0x30 [intel_qat]
adf_response_handler+0xbd/0x1a0 [intel_qat]
tasklet_action_common.constprop.0+0xcd/0xe0
__do_softirq+0xf8/0x30c
__irq_exit_rcu+0xbf/0x140
common_interrupt+0xb9/0xd0
</IRQ>
<TASK>
The original commit was correct.
Cc: <stable@vger.kernel.org>
Reported-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
When CONFIG_DMA_API_DEBUG is selected, while running the crypto self
test on the QAT crypto algorithms, the function add_dma_entry() reports
a warning similar to the one below, saying that overlapping mappings
are not supported. This occurs in tests where the input and the output
scatter list point to the same buffers (i.e. two different scatter lists
which point to the same chunks of memory).
The logic that implements the mapping uses the flag DMA_BIDIRECTIONAL
for both the input and the output scatter lists which leads to
overlapped write mappings. These are not supported by the DMA layer.
Fix by specifying the correct DMA transfer directions when mapping
buffers. For in-place operations where the input scatter list
matches the output scatter list, buffers are mapped once with
DMA_BIDIRECTIONAL, otherwise input buffers are mapped using the flag
DMA_TO_DEVICE and output buffers are mapped with DMA_FROM_DEVICE.
Overlapping a read mapping with a write mapping is a valid case in
dma-coherent devices like QAT.
The function that frees and unmaps the buffers, qat_alg_free_bufl()
has been changed accordingly to the changes to the mapping function.
DMA-API: 4xxx 0000:06:00.0: cacheline tracking EEXIST, overlapping mappings aren't supported
WARNING: CPU: 53 PID: 4362 at kernel/dma/debug.c:570 add_dma_entry+0x1e9/0x270
...
Call Trace:
dma_map_page_attrs+0x82/0x2d0
? preempt_count_add+0x6a/0xa0
qat_alg_sgl_to_bufl+0x45b/0x990 [intel_qat]
qat_alg_aead_dec+0x71/0x250 [intel_qat]
crypto_aead_decrypt+0x3d/0x70
test_aead_vec_cfg+0x649/0x810
? number+0x310/0x3a0
? vsnprintf+0x2a3/0x550
? scnprintf+0x42/0x70
? valid_sg_divisions.constprop.0+0x86/0xa0
? test_aead_vec+0xdf/0x120
test_aead_vec+0xdf/0x120
alg_test_aead+0x185/0x400
alg_test+0x3d8/0x500
? crypto_acomp_scomp_free_ctx+0x30/0x30
? __schedule+0x32a/0x12a0
? ttwu_queue_wakelist+0xbf/0x110
? _raw_spin_unlock_irqrestore+0x23/0x40
? try_to_wake_up+0x83/0x570
? _raw_spin_unlock_irqrestore+0x23/0x40
? __set_cpus_allowed_ptr_locked+0xea/0x1b0
? crypto_acomp_scomp_free_ctx+0x30/0x30
cryptomgr_test+0x27/0x50
kthread+0xe6/0x110
? kthread_complete_and_exit+0x20/0x20
ret_from_fork+0x1f/0x30
Fixes: d370cec ("crypto: qat - Intel(R) QAT crypto interface")
Link: https://lore.kernel.org/linux-crypto/20220223080400.139367-1-gilad@benyossef.com/
Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The QAT HW supports an hardware mechanism to detect an accelerator hang.
The reporting of a hang occurs after a watchdog timer (WDT) expires.
The value of the WDT set previously was too small and was causing false
positives.
Change the default value of the WDT to 0x7000000ULL to avoid this.
Fixes: 1c4d9d5bbb5a ("crypto: qat - enable detection of accelerators hang")
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Lucas Segarra Fernandez <lucas.segarra.fernandez@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Follow the advice of the below link and prefer 'strscpy' in this
subsystem. Conversion is 1:1 because the return value is not used.
Generated by a coccinelle script.
Link: https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/
Signed-off-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The function qat_uclo_check_image() validates the MMP and AE firmware
images. If the QAT device supports firmware authentication (indicated
by the handle to firmware loader), the input signed binary MMP and AE
images are validated by parsing the following information:
- Header length
- Full size of the binary
- Type of binary image (MMP or AE Firmware)
Firmware binaries use RSA3K for signing and verification.
The header length for the RSA3k is 0x384 bytes.
All the size field values in the binary are quantified
as DWORDS (1 DWORD = 4bytes).
On an invalid value the function prints an error message and returns
with an error code "EINVAL".
Signed-off-by: Srinivas Kerekare <srinivas.kerekare@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
qat_4xxx devices can be configured to allow either crypto or compression
operations. At the moment, devices are configured statically according to
the following rule:
- odd numbered devices assigned to compression services
- even numbered devices assigned to crypto services
Expose the sysfs attribute /sys/bus/pci/devices/<BDF>/qat/cfg_services
to allow to detect the configuration of a device and to change it.
The `cfg_service` attribute is only exposed for qat_4xxx devices and it
is limited to two configurations: (1) "sym;asym" for crypto services and
"dc" for compression services.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Co-developed-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Signed-off-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The function adf_sriov_prepare_restart() is used in adf_sriov.c to stop
and shutdown a device preserving its configuration.
Since this function will be re-used by the logic that allows to
reconfigure the device through sysfs, move it to adf_init.c and rename
it as adf_dev_shutdown_cache_cfg();
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The function adf_cfg_add_key_value_param() allows to insert duplicates
entries in the key value store of the driver.
Change the behaviour of that function to the following policy:
- if the key doesn't exist, add it;
- if the key already exists with a different value, then delete it and
replace it with a new one containing the new value;
- if the key exists with the same value, then return without doing
anything.
The behaviour of this function has been changed in order to easily
update key-values in the driver database. In particular this is required
to update the value of the ServiceEnables key used to change the service
loaded on a device.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Expose the device state through an attribute in sysfs and allow to
change it. This is to stop and shutdown a QAT device in order to change
its configuration.
The state attribute has been added to a newly created `qat` attribute
group which will contain all _QAT specific_ attributes.
The logic that implements the sysfs entries is part of a new file,
adf_sysfs.c. This exposes an entry point to allow the driver to create
attributes.
The function that creates the sysfs attributes is called from the probe
function of the driver and not in the state machine init function to
allow the change of states even if the device is in the down state.
In order to restore the device configuration between a transition from
down to up, the function that configures the devices has been abstracted
into the HW data structure.
The `state` attribute is only exposed for qat_4xxx devices.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Co-developed-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Signed-off-by: Tomasz Kowallik <tomaszx.kowalik@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Reviewed-by: Vladis Dronov <vdronov@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
This dependency looks outdated. After the previous patch, we have been able
to use this driver to encrypt some data and to create working VF on arm64.
We have not tested it yet on any big endian machine, hence the new dependency
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Currently the QAT driver code uses a self-defined wrapper function
called get_current_node() when it wants to learn the current NUMA node.
This implementation references the topology_physical_package_id[] array,
which more or less coincidentally contains the NUMA node id, at least
on x86.
Because this is not universal, and Linux offers a direct function to
learn the NUMA node ID, replace that function with a call to
numa_node_id(), which would work everywhere.
This fixes the QAT driver operation on arm64 machines.
Reported-by: Yoan Picchi <Yoan.Picchi@arm.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Yoan Picchi <yoan.picchi@arm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
QAT_401xx is a derivative of 4xxx. Add support for that device in the
qat_4xxx driver by including the DIDs (both PF and VF), extending the
probe and the firmware loader.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Srinivas Kerekare <srinivas.kerekare@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Re-enable the registration of algorithms after fixes to (1) use
pre-allocated buffers in the datapath and (2) support the
CRYPTO_TFM_REQ_MAY_BACKLOG flag.
This reverts commit 8893d27ffcaf6ec6267038a177cb87bcde4dd3de.
Cc: stable@vger.kernel.org
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Marco Chiappero <marco.chiappero@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
If a request has the flag CRYPTO_TFM_REQ_MAY_SLEEP set, allocate memory
using the flag GFP_KERNEL otherwise use GFP_ATOMIC.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Reject requests with a source buffer that is bigger than the size of the
key. This is to prevent a possible integer underflow that might happen
when copying the source scatterlist into a linear buffer.
Cc: stable@vger.kernel.org
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
Reject requests with a source buffer that is bigger than the size of the
key. This is to prevent a possible integer underflow that might happen
when copying the source scatterlist into a linear buffer.
Cc: stable@vger.kernel.org
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
The functions qat_dh_compute_value() allocates memory with
dma_alloc_coherent() if the source or the destination buffers are made
of multiple flat buffers or of a size that is not compatible with the
hardware.
This memory is then freed with dma_free_coherent() in the context of a
tasklet invoked to handle the response for the corresponding request.
According to Documentation/core-api/dma-api-howto.rst, the function
dma_free_coherent() cannot be called in an interrupt context.
Replace allocations with dma_alloc_coherent() in the function
qat_dh_compute_value() with kmalloc() + dma_map_single().
Cc: stable@vger.kernel.org
Fixes: c9839143ebbf ("crypto: qat - Add DH support")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
After commit f5ff79fddf0e ("dma-mapping: remove CONFIG_DMA_REMAP"), if
the algorithms are enabled, the driver crashes with a BUG_ON while
executing vunmap() in the context of a tasklet. This is due to the fact
that the function dma_free_coherent() cannot be called in an interrupt
context (see Documentation/core-api/dma-api-howto.rst).
The functions qat_rsa_enc() and qat_rsa_dec() allocate memory with
dma_alloc_coherent() if the source or the destination buffers are made
of multiple flat buffers or of a size that is not compatible with the
hardware.
This memory is then freed with dma_free_coherent() in the context of a
tasklet invoked to handle the response for the corresponding request.
Replace allocations with dma_alloc_coherent() in the functions
qat_rsa_enc() and qat_rsa_dec() with kmalloc() + dma_map_single().
Cc: stable@vger.kernel.org
Fixes: a990532023b9 ("crypto: qat - Add support for RSA algorithm")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Adam Guerin <adam.guerin@intel.com>
Reviewed-by: Wojciech Ziemba <wojciech.ziemba@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
