diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2023-06-29 11:00:17 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2023-06-29 11:00:17 -0700 |
| commit | 1b722407a13b7f8658d2e26917791f32805980a2 (patch) | |
| tree | 30aab582725a46e42843d75e2eb9ce4151f0f3ed /include | |
| parent | f8824e151fbfa0ac0a258015d606ea6f4a10251b (diff) | |
| parent | 5ff2977b19769fd24b0cfbe7cbe4d5114b6106af (diff) | |
Merge tag 'drm-next-2023-06-29' of git://anongit.freedesktop.org/drm/drm
Pull drm updates from Dave Airlie:
"There is one set of patches to misc for a i915 gsc/mei proxy driver.
Otherwise it's mostly amdgpu/i915/msm, lots of hw enablement and lots
of refactoring.
core:
- replace strlcpy with strscpy
- EDID changes to support further conversion to struct drm_edid
- Move i915 DSC parameter code to common DRM helpers
- Add Colorspace functionality
aperture:
- ignore framebuffers with non-primary devices
fbdev:
- use fbdev i/o helpers
- add Kconfig options for fb_ops helpers
- use new fb io helpers directly in drivers
sysfs:
- export DRM connector ID
scheduler:
- Avoid an infinite loop
ttm:
- store function table in .rodata
- Add query for TTM mem limit
- Add NUMA awareness to pools
- Export ttm_pool_fini()
bridge:
- fsl-ldb: support i.MX6SX
- lt9211, lt9611: remove blanking packets
- tc358768: implement input bus formats, devm cleanups
- ti-snd65dsi86: implement wait_hpd_asserted
- analogix: fix endless probe loop
- samsung-dsim: support swapped clock, fix enabling, support var
clock
- display-connector: Add support for external power supply
- imx: Fix module linking
- tc358762: Support reset GPIO
panel:
- nt36523: Support Lenovo J606F
- st7703: Support Anbernic RG353V-V2
- InnoLux G070ACE-L01 support
- boe-tv101wum-nl6: Improve initialization
- sharp-ls043t1le001: Mode fixes
- simple: BOE EV121WXM-N10-1850, S6D7AA0
- Ampire AM-800480L1TMQW-T00H
- Rocktech RK043FN48H
- Starry himax83102-j02
- Starry ili9882t
amdgpu:
- add new ctx query flag to handle reset better
- add new query/set shadow buffer for rdna3
- DCN 3.2/3.1.x/3.0.x updates
- Enable DC_FP on loongarch
- PCIe fix for RDNA2
- improve DC FAMS/SubVP support for better power management
- partition support for lots of engines
- Take NUMA into account when allocating memory
- Add new DRM_AMDGPU_WERROR config parameter to help with CI
- Initial SMU13 overdrive support
- Add support for new colorspace KMS API
- W=1 fixes
amdkfd:
- Query TTM mem limit rather than hardcoding it
- GC 9.4.3 partition support
- Handle NUMA for partitions
- Add debugger interface for enabling gdb
- Add KFD event age tracking
radeon:
- Fix possible UAF
i915:
- new getparam for PXP support
- GSC/MEI proxy driver
- Meteorlake display enablement
- avoid clearing preallocated framebuffers with TTM
- implement framebuffer mmap support
- Disable sampler indirect state in bindless heap
- Enable fdinfo for GuC backends
- GuC loading and firmware table handling fixes
- Various refactors for multi-tile enablement
- Define MOCS and PAT tables for MTL
- GSC/MEI support for Meteorlake
- PMU multi-tile support
- Large driver kernel doc cleanup
- Allow VRR toggling and arbitrary refresh rates
- Support async flips on linear buffers on display ver 12+
- Expose CRTC CTM property on ILK/SNB/VLV
- New debugfs for display clock frequencies
- Hotplug refactoring
- Display refactoring
- I915_GEM_CREATE_EXT_SET_PAT for Mesa on Meteorlake
- Use large rings for compute contexts
- HuC loading for MTL
- Allow user to set cache at BO creation
- MTL powermanagement enhancements
- Switch to dedicated workqueues to stop using flush_scheduled_work()
- Move display runtime init under display/
- Remove 10bit gamma on desktop gen3 parts, they don't support it
habanalabs:
- uapi: return 0 for user queries if there was a h/w or f/w error
- Add pci health check when we lose connection with the firmware.
This can be used to distinguish between pci link down and firmware
getting stuck.
- Add more info to the error print when TPC interrupt occur.
- Firmware fixes
msm:
- Adreno A660 bindings
- SM8350 MDSS bindings fix
- Added support for DPU on sm6350 and sm6375 platforms
- Implemented tearcheck support to support vsync on SM150 and newer
platforms
- Enabled missing features (DSPP, DSC, split display) on sc8180x,
sc8280xp, sm8450
- Added support for DSI and 28nm DSI PHY on MSM8226 platform
- Added support for DSI on sm6350 and sm6375 platforms
- Added support for display controller on MSM8226 platform
- A690 GPU support
- Move cmdstream dumping out of fence signaling path
- a610 support
- Support for a6xx devices without GMU
nouveau:
- NULL ptr before deref fixes
armada:
- implement fbdev emulation as client
sun4i:
- fix mipi-dsi dotclock
- release clocks
vc4:
- rgb range toggle property
- BT601 / BT2020 HDMI support
vkms:
- convert to drmm helpers
- add reflection and rotation support
- fix rgb565 conversion
gma500:
- fix iomem access
shmobile:
- support renesas soc platform
- enable fbdev
mxsfb:
- Add support for i.MX93 LCDIF
stm:
- dsi: Use devm_ helper
- ltdc: Fix potential invalid pointer deref
renesas:
- Group drivers in renesas subdirectory to prepare for new platform
- Drop deprecated R-Car H3 ES1.x support
meson:
- Add support for MIPI DSI displays
virtio:
- add sync object support
mediatek:
- Add display binding document for MT6795"
* tag 'drm-next-2023-06-29' of git://anongit.freedesktop.org/drm/drm: (1791 commits)
drm/i915: Fix a NULL vs IS_ERR() bug
drm/i915: make i915_drm_client_fdinfo() reference conditional again
drm/i915/huc: Fix missing error code in intel_huc_init()
drm/i915/gsc: take a wakeref for the proxy-init-completion check
drm/msm/a6xx: Add A610 speedbin support
drm/msm/a6xx: Add A619_holi speedbin support
drm/msm/a6xx: Use adreno_is_aXYZ macros in speedbin matching
drm/msm/a6xx: Use "else if" in GPU speedbin rev matching
drm/msm/a6xx: Fix some A619 tunables
drm/msm/a6xx: Add A610 support
drm/msm/a6xx: Add support for A619_holi
drm/msm/adreno: Disable has_cached_coherent in GMU wrapper configurations
drm/msm/a6xx: Introduce GMU wrapper support
drm/msm/a6xx: Move CX GMU power counter enablement to hw_init
drm/msm/a6xx: Extend and explain UBWC config
drm/msm/a6xx: Remove both GBIF and RBBM GBIF halt on hw init
drm/msm/a6xx: Add a helper for software-resetting the GPU
drm/msm/a6xx: Improve a6xx_bus_clear_pending_transactions()
drm/msm/a6xx: Move a6xx_bus_clear_pending_transactions to a6xx_gpu
drm/msm/a6xx: Move force keepalive vote removal to a6xx_gmu_force_off()
...
Diffstat (limited to 'include')
32 files changed, 1837 insertions, 198 deletions
diff --git a/include/asm-generic/fb.h b/include/asm-generic/fb.h index f9f18101ed36..bb7ee9c70e60 100644 --- a/include/asm-generic/fb.h +++ b/include/asm-generic/fb.h @@ -1,13 +1,135 @@ /* SPDX-License-Identifier: GPL-2.0 */ + #ifndef __ASM_GENERIC_FB_H_ #define __ASM_GENERIC_FB_H_ -#include <linux/fb.h> -#define fb_pgprotect(...) do {} while (0) +/* + * Only include this header file from your architecture's <asm/fb.h>. + */ + +#include <linux/io.h> +#include <linux/mm_types.h> +#include <linux/pgtable.h> + +struct fb_info; +struct file; + +#ifndef fb_pgprotect +#define fb_pgprotect fb_pgprotect +static inline void fb_pgprotect(struct file *file, struct vm_area_struct *vma, + unsigned long off) +{ + vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); +} +#endif +#ifndef fb_is_primary_device +#define fb_is_primary_device fb_is_primary_device static inline int fb_is_primary_device(struct fb_info *info) { return 0; } +#endif + +/* + * I/O helpers for the framebuffer. Prefer these functions over their + * regular counterparts. The regular I/O functions provide in-order + * access and swap bytes to/from little-endian ordering. Neither is + * required for framebuffers. Instead, the helpers read and write + * raw framebuffer data. Independent operations can be reordered for + * improved performance. + */ + +#ifndef fb_readb +static inline u8 fb_readb(const volatile void __iomem *addr) +{ + return __raw_readb(addr); +} +#define fb_readb fb_readb +#endif + +#ifndef fb_readw +static inline u16 fb_readw(const volatile void __iomem *addr) +{ + return __raw_readw(addr); +} +#define fb_readw fb_readw +#endif + +#ifndef fb_readl +static inline u32 fb_readl(const volatile void __iomem *addr) +{ + return __raw_readl(addr); +} +#define fb_readl fb_readl +#endif + +#ifndef fb_readq +#if defined(__raw_readq) +static inline u64 fb_readq(const volatile void __iomem *addr) +{ + return __raw_readq(addr); +} +#define fb_readq fb_readq +#endif +#endif + +#ifndef fb_writeb +static inline void fb_writeb(u8 b, volatile void __iomem *addr) +{ + __raw_writeb(b, addr); +} +#define fb_writeb fb_writeb +#endif + +#ifndef fb_writew +static inline void fb_writew(u16 b, volatile void __iomem *addr) +{ + __raw_writew(b, addr); +} +#define fb_writew fb_writew +#endif + +#ifndef fb_writel +static inline void fb_writel(u32 b, volatile void __iomem *addr) +{ + __raw_writel(b, addr); +} +#define fb_writel fb_writel +#endif + +#ifndef fb_writeq +#if defined(__raw_writeq) +static inline void fb_writeq(u64 b, volatile void __iomem *addr) +{ + __raw_writeq(b, addr); +} +#define fb_writeq fb_writeq +#endif +#endif + +#ifndef fb_memcpy_fromio +static inline void fb_memcpy_fromio(void *to, const volatile void __iomem *from, size_t n) +{ + memcpy_fromio(to, from, n); +} +#define fb_memcpy_fromio fb_memcpy_fromio +#endif + +#ifndef fb_memcpy_toio +static inline void fb_memcpy_toio(volatile void __iomem *to, const void *from, size_t n) +{ + memcpy_toio(to, from, n); +} +#define fb_memcpy_toio fb_memcpy_toio +#endif + +#ifndef fb_memset +static inline void fb_memset_io(volatile void __iomem *addr, int c, size_t n) +{ + memset_io(addr, c, n); +} +#define fb_memset fb_memset_io +#endif #endif /* __ASM_GENERIC_FB_H_ */ diff --git a/include/drm/bridge/samsung-dsim.h b/include/drm/bridge/samsung-dsim.h index ba5484de2b30..05100e91ecb9 100644 --- a/include/drm/bridge/samsung-dsim.h +++ b/include/drm/bridge/samsung-dsim.h @@ -54,11 +54,14 @@ struct samsung_dsim_driver_data { unsigned int has_freqband:1; unsigned int has_clklane_stop:1; unsigned int num_clks; + unsigned int min_freq; unsigned int max_freq; unsigned int wait_for_reset; unsigned int num_bits_resol; unsigned int pll_p_offset; const unsigned int *reg_values; + u16 m_min; + u16 m_max; }; struct samsung_dsim_host_ops { @@ -90,11 +93,14 @@ struct samsung_dsim { u32 pll_clk_rate; u32 burst_clk_rate; + u32 hs_clock; u32 esc_clk_rate; u32 lanes; u32 mode_flags; u32 format; + bool swap_dn_dp_clk; + bool swap_dn_dp_data; int state; struct drm_property *brightness; struct completion completed; diff --git a/include/drm/display/drm_dp.h b/include/drm/display/drm_dp.h index f8813c1e059b..02f2ac4dd2df 100644 --- a/include/drm/display/drm_dp.h +++ b/include/drm/display/drm_dp.h @@ -982,6 +982,7 @@ #define DP_EDP_GENERAL_CAP_2 0x703 # define DP_EDP_OVERDRIVE_ENGINE_ENABLED (1 << 0) +# define DP_EDP_PANEL_LUMINANCE_CONTROL_CAPABLE (1 << 4) #define DP_EDP_GENERAL_CAP_3 0x704 /* eDP 1.4 */ # define DP_EDP_X_REGION_CAP_MASK (0xf << 0) @@ -1007,6 +1008,7 @@ # define DP_EDP_DYNAMIC_BACKLIGHT_ENABLE (1 << 4) # define DP_EDP_REGIONAL_BACKLIGHT_ENABLE (1 << 5) # define DP_EDP_UPDATE_REGION_BRIGHTNESS (1 << 6) /* eDP 1.4 */ +# define DP_EDP_PANEL_LUMINANCE_CONTROL_ENABLE (1 << 7) #define DP_EDP_BACKLIGHT_BRIGHTNESS_MSB 0x722 #define DP_EDP_BACKLIGHT_BRIGHTNESS_LSB 0x723 @@ -1031,6 +1033,7 @@ #define DP_EDP_DBC_MINIMUM_BRIGHTNESS_SET 0x732 #define DP_EDP_DBC_MAXIMUM_BRIGHTNESS_SET 0x733 +#define DP_EDP_PANEL_TARGET_LUMINANCE_VALUE 0x734 #define DP_EDP_REGIONAL_BACKLIGHT_BASE 0x740 /* eDP 1.4 */ #define DP_EDP_REGIONAL_BACKLIGHT_0 0x741 /* eDP 1.4 */ @@ -1632,7 +1635,7 @@ enum dp_pixelformat { * * This enum is used to indicate DP VSC SDP Colorimetry formats. * It is based on DP 1.4 spec [Table 2-117: VSC SDP Payload for DB16 through - * DB18] and a name of enum member follows DRM_MODE_COLORIMETRY definition. + * DB18] and a name of enum member follows enum drm_colorimetry definition. * * @DP_COLORIMETRY_DEFAULT: sRGB (IEC 61966-2-1) or * ITU-R BT.601 colorimetry format diff --git a/include/drm/display/drm_dp_mst_helper.h b/include/drm/display/drm_dp_mst_helper.h index 32c764fb9cb5..ed5c9660563c 100644 --- a/include/drm/display/drm_dp_mst_helper.h +++ b/include/drm/display/drm_dp_mst_helper.h @@ -138,12 +138,7 @@ struct drm_dp_mst_port { * @cached_edid: for DP logical ports - make tiling work by ensuring * that the EDID for all connectors is read immediately. */ - struct edid *cached_edid; - /** - * @has_audio: Tracks whether the sink connector to this port is - * audio-capable. - */ - bool has_audio; + const struct drm_edid *cached_edid; /** * @fec_capable: bool indicating if FEC can be supported up to that @@ -815,8 +810,11 @@ void drm_dp_mst_topology_mgr_destroy(struct drm_dp_mst_topology_mgr *mgr); bool drm_dp_read_mst_cap(struct drm_dp_aux *aux, const u8 dpcd[DP_RECEIVER_CAP_SIZE]); int drm_dp_mst_topology_mgr_set_mst(struct drm_dp_mst_topology_mgr *mgr, bool mst_state); -int drm_dp_mst_hpd_irq(struct drm_dp_mst_topology_mgr *mgr, u8 *esi, bool *handled); - +int drm_dp_mst_hpd_irq_handle_event(struct drm_dp_mst_topology_mgr *mgr, + const u8 *esi, + u8 *ack, + bool *handled); +void drm_dp_mst_hpd_irq_send_new_request(struct drm_dp_mst_topology_mgr *mgr); int drm_dp_mst_detect_port(struct drm_connector *connector, @@ -824,7 +822,12 @@ drm_dp_mst_detect_port(struct drm_connector *connector, struct drm_dp_mst_topology_mgr *mgr, struct drm_dp_mst_port *port); -struct edid *drm_dp_mst_get_edid(struct drm_connector *connector, struct drm_dp_mst_topology_mgr *mgr, struct drm_dp_mst_port *port); +const struct drm_edid *drm_dp_mst_edid_read(struct drm_connector *connector, + struct drm_dp_mst_topology_mgr *mgr, + struct drm_dp_mst_port *port); +struct edid *drm_dp_mst_get_edid(struct drm_connector *connector, + struct drm_dp_mst_topology_mgr *mgr, + struct drm_dp_mst_port *port); int drm_dp_get_vc_payload_bw(const struct drm_dp_mst_topology_mgr *mgr, int link_rate, int link_lane_count); diff --git a/include/drm/display/drm_dsc_helper.h b/include/drm/display/drm_dsc_helper.h index 8b41edbbabab..913aa2071232 100644 --- a/include/drm/display/drm_dsc_helper.h +++ b/include/drm/display/drm_dsc_helper.h @@ -10,11 +10,24 @@ #include <drm/display/drm_dsc.h> +enum drm_dsc_params_type { + DRM_DSC_1_2_444, + DRM_DSC_1_1_PRE_SCR, /* legacy params from DSC 1.1 */ + DRM_DSC_1_2_422, + DRM_DSC_1_2_420, +}; + void drm_dsc_dp_pps_header_init(struct dp_sdp_header *pps_header); int drm_dsc_dp_rc_buffer_size(u8 rc_buffer_block_size, u8 rc_buffer_size); void drm_dsc_pps_payload_pack(struct drm_dsc_picture_parameter_set *pps_sdp, const struct drm_dsc_config *dsc_cfg); +void drm_dsc_set_const_params(struct drm_dsc_config *vdsc_cfg); +void drm_dsc_set_rc_buf_thresh(struct drm_dsc_config *vdsc_cfg); +int drm_dsc_setup_rc_params(struct drm_dsc_config *vdsc_cfg, enum drm_dsc_params_type type); int drm_dsc_compute_rc_parameters(struct drm_dsc_config *vdsc_cfg); +u8 drm_dsc_initial_scale_value(const struct drm_dsc_config *dsc); +u32 drm_dsc_flatness_det_thresh(const struct drm_dsc_config *dsc); +u32 drm_dsc_get_bpp_int(const struct drm_dsc_config *vdsc_cfg); #endif /* _DRM_DSC_HELPER_H_ */ diff --git a/include/drm/drm_aperture.h b/include/drm/drm_aperture.h index 7096703c3949..cbe33b49fd5d 100644 --- a/include/drm/drm_aperture.h +++ b/include/drm/drm_aperture.h @@ -13,14 +13,13 @@ int devm_aperture_acquire_from_firmware(struct drm_device *dev, resource_size_t resource_size_t size); int drm_aperture_remove_conflicting_framebuffers(resource_size_t base, resource_size_t size, - bool primary, const struct drm_driver *req_driver); + const struct drm_driver *req_driver); int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, const struct drm_driver *req_driver); /** * drm_aperture_remove_framebuffers - remove all existing framebuffers - * @primary: also kick vga16fb if present * @req_driver: requesting DRM driver * * This function removes all graphics device drivers. Use this function on systems @@ -30,9 +29,9 @@ int drm_aperture_remove_conflicting_pci_framebuffers(struct pci_dev *pdev, * 0 on success, or a negative errno code otherwise */ static inline int -drm_aperture_remove_framebuffers(bool primary, const struct drm_driver *req_driver) +drm_aperture_remove_framebuffers(const struct drm_driver *req_driver) { - return drm_aperture_remove_conflicting_framebuffers(0, (resource_size_t)-1, primary, + return drm_aperture_remove_conflicting_framebuffers(0, (resource_size_t)-1, req_driver); } diff --git a/include/drm/drm_connector.h b/include/drm/drm_connector.h index 7b5048516185..d300fde6c1a4 100644 --- a/include/drm/drm_connector.h +++ b/include/drm/drm_connector.h @@ -30,6 +30,7 @@ #include <linux/notifier.h> #include <drm/drm_mode_object.h> #include <drm/drm_util.h> +#include <drm/drm_property.h> #include <uapi/drm/drm_mode.h> @@ -199,6 +200,11 @@ enum drm_connector_tv_mode { */ DRM_MODE_TV_MODE_SECAM, + /** + * @DRM_MODE_TV_MODE_MAX: Number of analog TV output modes. + * + * Internal implementation detail; this is not uABI. + */ DRM_MODE_TV_MODE_MAX, }; @@ -419,37 +425,106 @@ enum drm_privacy_screen_status { PRIVACY_SCREEN_ENABLED_LOCKED, }; -/* - * This is a consolidated colorimetry list supported by HDMI and +/** + * enum drm_colorspace - color space + * + * This enum is a consolidated colorimetry list supported by HDMI and * DP protocol standard. The respective connectors will register * a property with the subset of this list (supported by that * respective protocol). Userspace will set the colorspace through * a colorspace property which will be created and exposed to * userspace. + * + * DP definitions come from the DP v2.0 spec + * HDMI definitions come from the CTA-861-H spec + * + * A note on YCC and RGB variants: + * + * Since userspace is not aware of the encoding on the wire + * (RGB or YCbCr), drivers are free to pick the appropriate + * variant, regardless of what userspace selects. E.g., if + * BT2020_RGB is selected by userspace a driver will pick + * BT2020_YCC if the encoding on the wire is YUV444 or YUV420. + * + * @DRM_MODE_COLORIMETRY_DEFAULT: + * Driver specific behavior. + * @DRM_MODE_COLORIMETRY_NO_DATA: + * Driver specific behavior. + * @DRM_MODE_COLORIMETRY_SMPTE_170M_YCC: + * (HDMI) + * SMPTE ST 170M colorimetry format + * @DRM_MODE_COLORIMETRY_BT709_YCC: + * (HDMI, DP) + * ITU-R BT.709 colorimetry format + * @DRM_MODE_COLORIMETRY_XVYCC_601: + * (HDMI, DP) + * xvYCC601 colorimetry format + * @DRM_MODE_COLORIMETRY_XVYCC_709: + * (HDMI, DP) + * xvYCC709 colorimetry format + * @DRM_MODE_COLORIMETRY_SYCC_601: + * (HDMI, DP) + * sYCC601 colorimetry format + * @DRM_MODE_COLORIMETRY_OPYCC_601: + * (HDMI, DP) + * opYCC601 colorimetry format + * @DRM_MODE_COLORIMETRY_OPRGB: + * (HDMI, DP) + * opRGB colorimetry format + * @DRM_MODE_COLORIMETRY_BT2020_CYCC: + * (HDMI, DP) + * ITU-R BT.2020 Y'c C'bc C'rc (constant luminance) colorimetry format + * @DRM_MODE_COLORIMETRY_BT2020_RGB: + * (HDMI, DP) + * ITU-R BT.2020 R' G' B' colorimetry format + * @DRM_MODE_COLORIMETRY_BT2020_YCC: + * (HDMI, DP) + * ITU-R BT.2020 Y' C'b C'r colorimetry format + * @DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65: + * (HDMI) + * SMPTE ST 2113 P3D65 colorimetry format + * @DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER: + * (HDMI) + * SMPTE ST 2113 P3DCI colorimetry format + * @DRM_MODE_COLORIMETRY_RGB_WIDE_FIXED: + * (DP) + * RGB wide gamut fixed point colorimetry format + * @DRM_MODE_COLORIMETRY_RGB_WIDE_FLOAT: + * (DP) + * RGB wide gamut floating point + * (scRGB (IEC 61966-2-2)) colorimetry format + * @DRM_MODE_COLORIMETRY_BT601_YCC: + * (DP) + * ITU-R BT.601 colorimetry format + * The DP spec does not say whether this is the 525 or the 625 + * line version. */ - -/* For Default case, driver will set the colorspace */ -#define DRM_MODE_COLORIMETRY_DEFAULT 0 -/* CEA 861 Normal Colorimetry options */ -#define DRM_MODE_COLORIMETRY_NO_DATA 0 -#define DRM_MODE_COLORIMETRY_SMPTE_170M_YCC 1 -#define DRM_MODE_COLORIMETRY_BT709_YCC 2 -/* CEA 861 Extended Colorimetry Options */ -#define DRM_MODE_COLORIMETRY_XVYCC_601 3 -#define DRM_MODE_COLORIMETRY_XVYCC_709 4 -#define DRM_MODE_COLORIMETRY_SYCC_601 5 -#define DRM_MODE_COLORIMETRY_OPYCC_601 6 -#define DRM_MODE_COLORIMETRY_OPRGB 7 -#define DRM_MODE_COLORIMETRY_BT2020_CYCC 8 -#define DRM_MODE_COLORIMETRY_BT2020_RGB 9 -#define DRM_MODE_COLORIMETRY_BT2020_YCC 10 -/* Additional Colorimetry extension added as part of CTA 861.G */ -#define DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65 11 -#define DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER 12 -/* Additional Colorimetry Options added for DP 1.4a VSC Colorimetry Format */ -#define DRM_MODE_COLORIMETRY_RGB_WIDE_FIXED 13 -#define DRM_MODE_COLORIMETRY_RGB_WIDE_FLOAT 14 -#define DRM_MODE_COLORIMETRY_BT601_YCC 15 +enum drm_colorspace { + /* For Default case, driver will set the colorspace */ + DRM_MODE_COLORIMETRY_DEFAULT = 0, + /* CEA 861 Normal Colorimetry options */ + DRM_MODE_COLORIMETRY_NO_DATA = 0, + DRM_MODE_COLORIMETRY_SMPTE_170M_YCC = 1, + DRM_MODE_COLORIMETRY_BT709_YCC = 2, + /* CEA 861 Extended Colorimetry Options */ + DRM_MODE_COLORIMETRY_XVYCC_601 = 3, + DRM_MODE_COLORIMETRY_XVYCC_709 = 4, + DRM_MODE_COLORIMETRY_SYCC_601 = 5, + DRM_MODE_COLORIMETRY_OPYCC_601 = 6, + DRM_MODE_COLORIMETRY_OPRGB = 7, + DRM_MODE_COLORIMETRY_BT2020_CYCC = 8, + DRM_MODE_COLORIMETRY_BT2020_RGB = 9, + DRM_MODE_COLORIMETRY_BT2020_YCC = 10, + /* Additional Colorimetry extension added as part of CTA 861.G */ + DRM_MODE_COLORIMETRY_DCI_P3_RGB_D65 = 11, + DRM_MODE_COLORIMETRY_DCI_P3_RGB_THEATER = 12, + /* Additional Colorimetry Options added for DP 1.4a VSC Colorimetry Format */ + DRM_MODE_COLORIMETRY_RGB_WIDE_FIXED = 13, + DRM_MODE_COLORIMETRY_RGB_WIDE_FLOAT = 14, + DRM_MODE_COLORIMETRY_BT601_YCC = 15, + /* not a valid value; merely used for counting */ + DRM_MODE_COLORIMETRY_COUNT +}; /** * enum drm_bus_flags - bus_flags info for &drm_display_info @@ -654,6 +729,14 @@ struct drm_display_info { bool is_hdmi; /** + * @has_audio: True if the sink supports audio. + * + * This field shall be used instead of calling + * drm_detect_monitor_audio() when possible. + */ + bool has_audio; + + /** * @has_hdmi_infoframe: Does the sink support the HDMI infoframe? */ bool has_hdmi_infoframe; @@ -901,7 +984,7 @@ struct drm_connector_state { * colorspace change on Sink. This is most commonly used to switch * to wider color gamuts like BT2020. */ - u32 colorspace; + enum drm_colorspace colorspace; /** * @writeback_job: Writeback job for writeback connectors @@ -1925,8 +2008,10 @@ int drm_connector_attach_hdr_output_metadata_property(struct drm_connector *conn bool drm_connector_atomic_hdr_metadata_equal(struct drm_connector_state *old_state, struct drm_connector_state *new_state); int drm_mode_create_aspect_ratio_property(struct drm_device *dev); -int drm_mode_create_hdmi_colorspace_property(struct drm_connector *connector); -int drm_mode_create_dp_colorspace_property(struct drm_connector *connector); +int drm_mode_create_hdmi_colorspace_property(struct drm_connector *connector, + u32 supported_colorspaces); +int drm_mode_create_dp_colorspace_property(struct drm_connector *connector, + u32 supported_colorspaces); int drm_mode_create_content_type_property(struct drm_device *dev); int drm_mode_create_suggested_offset_properties(struct drm_device *dev); @@ -2009,6 +2094,7 @@ void drm_connector_list_iter_end(struct drm_connector_list_iter *iter); bool drm_connector_has_possible_encoder(struct drm_connector *connector, struct drm_encoder *encoder); +const char *drm_get_colorspace_name(enum drm_colorspace colorspace); /** * drm_for_each_connector_iter - connector_list iterator macro diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index b419c59c4bef..89e2706cac56 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -401,6 +401,13 @@ struct drm_driver { struct drm_device *dev, uint32_t handle, uint64_t *offset); + /** + * @show_fdinfo: + * + * Print device specific fdinfo. See Documentation/gpu/drm-usage-stats.rst. + */ + void (*show_fdinfo)(struct drm_printer *p, struct drm_file *f); + /** @major: driver major number */ int major; /** @minor: driver minor number */ diff --git a/include/drm/drm_edid.h b/include/drm/drm_edid.h index 571885d32907..169755d3de19 100644 --- a/include/drm/drm_edid.h +++ b/include/drm/drm_edid.h @@ -613,6 +613,8 @@ const struct drm_edid *drm_edid_read_ddc(struct drm_connector *connector, const struct drm_edid *drm_edid_read_custom(struct drm_connector *connector, int (*read_block)(void *context, u8 *buf, unsigned int block, size_t len), void *context); +const struct drm_edid *drm_edid_read_switcheroo(struct drm_connector *connector, + struct i2c_adapter *adapter); int drm_edid_connector_update(struct drm_connector *connector, const struct drm_edid *edid); int drm_edid_connector_add_modes(struct drm_connector *connector); diff --git a/include/drm/drm_fb_helper.h b/include/drm/drm_fb_helper.h index 72032c354a30..4863b0f8299e 100644 --- a/include/drm/drm_fb_helper.h +++ b/include/drm/drm_fb_helper.h @@ -253,31 +253,10 @@ void drm_fb_helper_fill_info(struct fb_info *info, struct drm_fb_helper *fb_helper, struct drm_fb_helper_surface_size *sizes); -void drm_fb_helper_deferred_io(struct fb_info *info, struct list_head *pagereflist); +void drm_fb_helper_damage_range(struct fb_info *info, off_t off, size_t len); +void drm_fb_helper_damage_area(struct fb_info *info, u32 x, u32 y, u32 width, u32 height); -ssize_t drm_fb_helper_sys_read(struct fb_info *info, char __user *buf, - size_t count, loff_t *ppos); -ssize_t drm_fb_helper_sys_write(struct fb_info *info, const char __user *buf, - size_t count, loff_t *ppos); - -void drm_fb_helper_sys_fillrect(struct fb_info *info, - const struct fb_fillrect *rect); -void drm_fb_helper_sys_copyarea(struct fb_info *info, - const struct fb_copyarea *area); -void drm_fb_helper_sys_imageblit(struct fb_info *info, - const struct fb_image *image); - -ssize_t drm_fb_helper_cfb_read(struct fb_info *info, char __user *buf, - size_t count, loff_t *ppos); -ssize_t drm_fb_helper_cfb_write(struct fb_info *info, const char __user *buf, - size_t count, loff_t *ppos); - -void drm_fb_helper_cfb_fillrect(struct fb_info *info, - const struct fb_fillrect *rect); -void drm_fb_helper_cfb_copyarea(struct fb_info *info, - const struct fb_copyarea *area); -void drm_fb_helper_cfb_imageblit(struct fb_info *info, - const struct fb_image *image); +void drm_fb_helper_deferred_io(struct fb_info *info, struct list_head *pagereflist); void drm_fb_helper_set_suspend(struct drm_fb_helper *fb_helper, bool suspend); void drm_fb_helper_set_suspend_unlocked(struct drm_fb_helper *fb_helper, @@ -394,62 +373,6 @@ static inline int drm_fb_helper_defio_init(struct drm_fb_helper *fb_helper) return -ENODEV; } -static inline ssize_t drm_fb_helper_sys_read(struct fb_info *info, - char __user *buf, size_t count, - loff_t *ppos) -{ - return -ENODEV; -} - -static inline ssize_t drm_fb_helper_sys_write(struct fb_info *info, - const char __user *buf, - size_t count, loff_t *ppos) -{ - return -ENODEV; -} - -static inline void drm_fb_helper_sys_fillrect(struct fb_info *info, - const struct fb_fillrect *rect) -{ -} - -static inline void drm_fb_helper_sys_copyarea(struct fb_info *info, - const struct fb_copyarea *area) -{ -} - -static inline void drm_fb_helper_sys_imageblit(struct fb_info *info, - const struct fb_image *image) -{ -} - -static inline ssize_t drm_fb_helper_cfb_read(struct fb_info *info, char __user *buf, - size_t count, loff_t *ppos) -{ - return -ENODEV; -} - -static inline ssize_t drm_fb_helper_cfb_write(struct fb_info *info, const char __user *buf, - size_t count, loff_t *ppos) -{ - return -ENODEV; -} - -static inline void drm_fb_helper_cfb_fillrect(struct fb_info *info, - const struct fb_fillrect *rect) -{ -} - -static inline void drm_fb_helper_cfb_copyarea(struct fb_info *info, - const struct fb_copyarea *area) -{ -} - -static inline void drm_fb_helper_cfb_imageblit(struct fb_info *info, - const struct fb_image *image) -{ -} - static inline void drm_fb_helper_set_suspend(struct drm_fb_helper *fb_helper, bool suspend) { diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index ecffe24e2b1b..966912053cb0 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -41,6 +41,7 @@ struct dma_fence; struct drm_file; struct drm_device; +struct drm_printer; struct device; struct file; @@ -258,6 +259,9 @@ struct drm_file { /** @pid: Process that opened this file. */ struct pid *pid; + /** @client_id: A unique id for fdinfo */ + u64 client_id; + /** @magic: Authentication magic, see @authenticated. */ drm_magic_t magic; @@ -439,6 +443,34 @@ void drm_send_event_timestamp_locked(struct drm_device *dev, struct drm_pending_event *e, ktime_t timestamp); +/** + * struct drm_memory_stats - GEM object stats associated + * @shared: Total size of GEM objects shared between processes + * @private: Total size of GEM objects + * @resident: Total size of GEM objects backing pages + * @purgeable: Total size of GEM objects that can be purged (resident and not active) + * @active: Total size of GEM objects active on one or more engines + * + * Used by drm_print_memory_stats() + */ +struct drm_memory_stats { + u64 shared; + u64 private; + u64 resident; + u64 purgeable; + u64 active; +}; + +enum drm_gem_object_status; + +void drm_print_memory_stats(struct drm_printer *p, + const struct drm_memory_stats *stats, + enum drm_gem_object_status supported_status, + const char *region); + +void drm_show_memory_stats(struct drm_printer *p, struct drm_file *file); +void drm_show_fdinfo(struct seq_file *m, struct file *f); + struct file *mock_drm_getfile(struct drm_minor *minor, unsigned int flags); #endif /* _DRM_FILE_H_ */ diff --git a/include/drm/drm_fixed.h b/include/drm/drm_fixed.h index 255645c1f9a8..6ea339d5de08 100644 --- a/include/drm/drm_fixed.h +++ b/include/drm/drm_fixed.h @@ -71,6 +71,7 @@ static inline u32 dfixed_div(fixed20_12 A, fixed20_12 B) } #define DRM_FIXED_POINT 32 +#define DRM_FIXED_POINT_HALF 16 #define DRM_FIXED_ONE (1ULL << DRM_FIXED_POINT) #define DRM_FIXED_DECIMAL_MASK (DRM_FIXED_ONE - 1) #define DRM_FIXED_DIGITS_MASK (~DRM_FIXED_DECIMAL_MASK) @@ -87,6 +88,11 @@ static inline int drm_fixp2int(s64 a) return ((s64)a) >> DRM_FIXED_POINT; } +static inline int drm_fixp2int_round(s64 a) +{ + return drm_fixp2int(a + (1 << (DRM_FIXED_POINT_HALF - 1))); +} + static inline int drm_fixp2int_ceil(s64 a) { if (a > 0) diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h index b8efd836edef..bbc721870c13 100644 --- a/include/drm/drm_gem.h +++ b/include/drm/drm_gem.h @@ -43,6 +43,25 @@ struct iosys_map; struct drm_gem_object; /** + * enum drm_gem_object_status - bitmask of object state for fdinfo reporting + * @DRM_GEM_OBJECT_RESIDENT: object is resident in memory (ie. not unpinned) + * @DRM_GEM_OBJECT_PURGEABLE: object marked as purgeable by userspace + * + * Bitmask of status used for fdinfo memory stats, see &drm_gem_object_funcs.status + * and drm_show_fdinfo(). Note that an object can DRM_GEM_OBJECT_PURGEABLE if + * it still active or not resident, in which case drm_show_fdinfo() will not + * account for it as purgeable. So drivers do not need to check if the buffer + * is idle and resident to return this bit. (Ie. userspace can mark a buffer + * as purgeable even while it is still busy on the GPU.. it does not _actually_ + * become puregeable until it becomes idle. The status gem object func does + * not need to consider this.) + */ +enum drm_gem_object_status { + DRM_GEM_OBJECT_RESIDENT = BIT(0), + DRM_GEM_OBJECT_PURGEABLE = BIT(1), +}; + +/** * struct drm_gem_object_funcs - GEM object functions */ struct drm_gem_object_funcs { @@ -175,6 +194,19 @@ struct drm_gem_object_funcs { int (*evict)(struct drm_gem_object *obj); /** + * @status: + * + * The optional status callback can return additional object state + * which determines which stats the object is counted against. The + * callback is called under table_lock. Racing against object status + * change is "harmless", and the callback can expect to not race + * against object destruction. + * + * Called by drm_show_memory_stats(). + */ + enum drm_gem_object_status (*status)(struct drm_gem_object *obj); + + /** * @vm_ops: * * Virtual memory operations used with mmap. diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index c0586d832260..e95b4837e5a3 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -201,7 +201,7 @@ struct drm_sched_entity { * by the scheduler thread, can be accessed locklessly from * drm_sched_job_arm() iff the queue is empty. */ - struct dma_fence *last_scheduled; + struct dma_fence __rcu *last_scheduled; /** * @last_user: last group leader pushing a job into the entity. @@ -549,7 +549,7 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity, unsigned int num_sched_list); void drm_sched_job_cleanup(struct drm_sched_job *job); -void drm_sched_wakeup(struct drm_gpu_scheduler *sched); +void drm_sched_wakeup_if_can_queue(struct drm_gpu_scheduler *sched); void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad); void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery); void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched); @@ -581,6 +581,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job); void drm_sched_entity_set_priority(struct drm_sched_entity *entity, enum drm_sched_priority priority); bool drm_sched_entity_is_ready(struct drm_sched_entity *entity); +int drm_sched_entity_error(struct drm_sched_entity *entity); void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence, struct dma_fence *fence); @@ -591,7 +592,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, void drm_sched_fence_free(struct drm_sched_fence *fence); void drm_sched_fence_scheduled(struct drm_sched_fence *fence); -void drm_sched_fence_finished(struct drm_sched_fence *fence); +void drm_sched_fence_finished(struct drm_sched_fence *fence, int result); unsigned long drm_sched_suspend_timeout(struct drm_gpu_scheduler *sched); void drm_sched_resume_timeout(struct drm_gpu_scheduler *sched, diff --git a/include/drm/i915_component.h b/include/drm/i915_component.h index c1e2a43d2d1e..56a84ee1c64c 100644 --- a/include/drm/i915_component.h +++ b/include/drm/i915_component.h @@ -29,7 +29,8 @@ enum i915_component_type { I915_COMPONENT_AUDIO = 1, I915_COMPONENT_HDCP, - I915_COMPONENT_PXP + I915_COMPONENT_PXP, + I915_COMPONENT_GSC_PROXY, }; /* MAX_PORT is the number of port diff --git a/include/drm/i915_drm.h b/include/drm/i915_drm.h index 7adce327c1c2..adff68538484 100644 --- a/include/drm/i915_drm.h +++ b/include/drm/i915_drm.h @@ -42,7 +42,7 @@ extern struct resource intel_graphics_stolen_res; * The Bridge device's PCI config space has information about the * fb aperture size and the amount of pre-reserved memory. * This is all handled in the intel-gtt.ko module. i915.ko only - * cares about the vga bit for the vga rbiter. + * cares about the vga bit for the vga arbiter. */ #define INTEL_GMCH_CTRL 0x52 #define INTEL_GMCH_VGA_DISABLE (1 << 1) diff --git a/include/drm/i915_gsc_proxy_mei_interface.h b/include/drm/i915_gsc_proxy_mei_interface.h new file mode 100644 index 000000000000..9462341d3ae1 --- /dev/null +++ b/include/drm/i915_gsc_proxy_mei_interface.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Copyright (c) 2022-2023 Intel Corporation + */ + +#ifndef _I915_GSC_PROXY_MEI_INTERFACE_H_ +#define _I915_GSC_PROXY_MEI_INTERFACE_H_ + +#include <linux/types.h> + +struct device; +struct module; + +/** + * struct i915_gsc_proxy_component_ops - ops for GSC Proxy services. + * @owner: Module providing the ops + * @send: sends a proxy message from GSC FW to ME FW + * @recv: receives a proxy message for GSC FW from ME FW + */ +struct i915_gsc_proxy_component_ops { + struct module *owner; + + /** + * send - Sends a proxy message to ME FW. + * @dev: device struct corresponding to the mei device + * @buf: message buffer to send + * @size: size of the message + * Return: bytes sent on success, negative errno value on failure + */ + int (*send)(struct device *dev, const void *buf, size_t size); + + /** + * recv - Receives a proxy message from ME FW. + * @dev: device struct corresponding to the mei device + * @buf: message buffer to contain the received message + * @size: size of the buffer + * Return: bytes received on success, negative errno value on failure + */ + int (*recv)(struct device *dev, void *buf, size_t size); +}; + +/** + * struct i915_gsc_proxy_component - Used for communication between i915 and + * MEI drivers for GSC proxy services + * @mei_dev: device that provide the GSC proxy service. + * @ops: Ops implemented by GSC proxy driver, used by i915 driver. + */ +struct i915_gsc_proxy_component { + struct device *mei_dev; + const struct i915_gsc_proxy_component_ops *ops; +}; + +#endif /* _I915_GSC_PROXY_MEI_INTERFACE_H_ */ diff --git a/include/drm/i915_hdcp_interface.h b/include/drm/i915_hdcp_interface.h index 2059b066f8a1..4c9c8167c2d5 100644 --- a/include/drm/i915_hdcp_interface.h +++ b/include/drm/i915_hdcp_interface.h @@ -168,12 +168,12 @@ struct i915_hdcp_ops { }; /** - * struct i915_hdcp_master - Used for communication between i915 + * struct i915_hdcp_arbiter - Used for communication between i915 * and hdcp drivers for the HDCP2.2 services * @hdcp_dev: device that provide the HDCP2.2 service from MEI Bus. * @hdcp_ops: Ops implemented by hdcp driver or intel_hdcp_gsc , used by i915 driver. */ -struct i915_hdcp_master { +struct i915_hdcp_arbiter { struct device *hdcp_dev; const struct i915_hdcp_ops *ops; diff --git a/include/drm/ttm/ttm_device.h b/include/drm/ttm/ttm_device.h index 56e82ba2d046..c22f30535c84 100644 --- a/include/drm/ttm/ttm_device.h +++ b/include/drm/ttm/ttm_device.h @@ -223,7 +223,7 @@ struct ttm_device { * @funcs: Function table for the device. * Constant after bo device init */ - struct ttm_device_funcs *funcs; + const struct ttm_device_funcs *funcs; /** * @sysman: Resource manager for the system domain. @@ -287,7 +287,7 @@ static inline void ttm_set_driver_manager(struct ttm_device *bdev, int type, bdev->man_drv[type] = manager; } -int ttm_device_init(struct ttm_device *bdev, struct ttm_device_funcs *funcs, +int ttm_device_init(struct ttm_device *bdev, const struct ttm_device_funcs *funcs, struct device *dev, struct address_space *mapping, struct drm_vma_offset_manager *vma_manager, bool use_dma_alloc, bool use_dma32); diff --git a/include/drm/ttm/ttm_pool.h b/include/drm/ttm/ttm_pool.h index 8ce14f9d202a..30a347e5aa11 100644 --- a/include/drm/ttm/ttm_pool.h +++ b/include/drm/ttm/ttm_pool.h @@ -61,12 +61,14 @@ struct ttm_pool_type { * struct ttm_pool - Pool for all caching and orders * * @dev: the device we allocate pages for + * @nid: which numa node to use * @use_dma_alloc: if coherent DMA allocations should be used * @use_dma32: if GFP_DMA32 should be used * @caching: pools for each caching/order */ struct ttm_pool { struct device *dev; + int nid; bool use_dma_alloc; bool use_dma32; @@ -81,7 +83,7 @@ int ttm_pool_alloc(struct ttm_pool *pool, struct ttm_tt *tt, void ttm_pool_free(struct ttm_pool *pool, struct ttm_tt *tt); void ttm_pool_init(struct ttm_pool *pool, struct device *dev, - bool use_dma_alloc, bool use_dma32); + int nid, bool use_dma_alloc, bool use_dma32); void ttm_pool_fini(struct ttm_pool *pool); int ttm_pool_debugfs(struct ttm_pool *pool, struct seq_file *m); diff --git a/include/drm/ttm/ttm_tt.h b/include/drm/ttm/ttm_tt.h index 977ca195a536..a4eff85b1f44 100644 --- a/include/drm/ttm/ttm_tt.h +++ b/include/drm/ttm/ttm_tt.h @@ -222,7 +222,7 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages); struct ttm_kmap_iter *ttm_kmap_iter_tt_init(struct ttm_kmap_iter_tt *iter_tt, struct ttm_tt *tt); - +unsigned long ttm_tt_pages_limit(void); #if IS_ENABLED(CONFIG_AGP) #include <linux/agp_backend.h> diff --git a/include/linux/aperture.h b/include/linux/aperture.h index 442f15a57cad..1a9a88b11584 100644 --- a/include/linux/aperture.h +++ b/include/linux/aperture.h @@ -14,7 +14,9 @@ int devm_aperture_acquire_for_platform_device(struct platform_device *pdev, resource_size_t size); int aperture_remove_conflicting_devices(resource_size_t base, resource_size_t size, - bool primary, const char *name); + const char *name); + +int __aperture_remove_legacy_vga_devices(struct pci_dev *pdev); int aperture_remove_conflicting_pci_devices(struct pci_dev *pdev, const char *name); #else @@ -26,7 +28,12 @@ static inline int devm_aperture_acquire_for_platform_device(struct platform_devi } static inline int aperture_remove_conflicting_devices(resource_size_t base, resource_size_t size, - bool primary, const char *name) + const char *name) +{ + return 0; +} + +static inline int __aperture_remove_legacy_vga_devices(struct pci_dev *pdev) { return 0; } @@ -39,7 +46,6 @@ static inline int aperture_remove_conflicting_pci_devices(struct pci_dev *pdev, /** * aperture_remove_all_conflicting_devices - remove all existing framebuffers - * @primary: also kick vga16fb if present; only relevant for VGA devices * @name: a descriptive name of the requesting driver * * This function removes all graphics device drivers. Use this function on systems @@ -48,9 +54,9 @@ static inline int aperture_remove_conflicting_pci_devices(struct pci_dev *pdev, * Returns: * 0 on success, or a negative errno code otherwise */ -static inline int aperture_remove_all_conflicting_devices(bool primary, const char *name) +static inline int aperture_remove_all_conflicting_devices(const char *name) { - return aperture_remove_conflicting_devices(0, (resource_size_t)-1, primary, name); + return aperture_remove_conflicting_devices(0, (resource_size_t)-1, name); } #endif diff --git a/include/linux/fb.h b/include/linux/fb.h index 08cb47da71f8..ce7d588edc3e 100644 --- a/include/linux/fb.h +++ b/include/linux/fb.h @@ -15,7 +15,8 @@ #include <linux/list.h> #include <linux/backlight.h> #include <linux/slab.h> -#include <asm/io.h> + +#include <asm/fb.h> struct vm_area_struct; struct fb_info; @@ -511,58 +512,6 @@ struct fb_info { */ #define STUPID_ACCELF_TEXT_SHIT -// This will go away -#if defined(__sparc__) - -/* We map all of our framebuffers such that big-endian accesses - * are what we want, so the following is sufficient. - */ - -// This will go away -#define fb_readb sbus_readb -#define fb_readw sbus_readw -#define fb_readl sbus_readl -#define fb_readq sbus_readq -#define fb_writeb sbus_writeb -#define fb_writew sbus_writew -#define fb_writel sbus_writel -#define fb_writeq sbus_writeq -#define fb_memset sbus_memset_io -#define fb_memcpy_fromfb sbus_memcpy_fromio -#define fb_memcpy_tofb sbus_memcpy_toio - -#elif defined(__i386__) || defined(__alpha__) || defined(__x86_64__) || \ - defined(__hppa__) || defined(__sh__) || defined(__powerpc__) || \ - defined(__arm__) || defined(__aarch64__) || defined(__mips__) - -#define fb_readb __raw_readb -#define fb_readw __raw_readw -#define fb_readl __raw_readl -#define fb_readq __raw_readq -#define fb_writeb __raw_writeb -#define fb_writew __raw_writew -#define fb_writel __raw_writel -#define fb_writeq __raw_writeq -#define fb_memset memset_io -#define fb_memcpy_fromfb memcpy_fromio -#define fb_memcpy_tofb memcpy_toio - -#else - -#define fb_readb(addr) (*(volatile u8 *) (addr)) -#define fb_readw(addr) (*(volatile u16 *) (addr)) -#define fb_readl(addr) (*(volatile u32 *) (addr)) -#define fb_readq(addr) (*(volatile u64 *) (addr)) -#define fb_writeb(b,addr) (*(volatile u8 *) (addr) = (b)) -#define fb_writew(b,addr) (*(volatile u16 *) (addr) = (b)) -#define fb_writel(b,addr) (*(volatile u32 *) (addr) = (b)) -#define fb_writeq(b,addr) (*(volatile u64 *) (addr) = (b)) -#define fb_memset memset -#define fb_memcpy_fromfb memcpy -#define fb_memcpy_tofb memcpy - -#endif - #define FB_LEFT_POS(p, bpp) (fb_be_math(p) ? (32 - (bpp)) : 0) #define FB_SHIFT_HIGH(p, val, bits) (fb_be_math(p) ? (val) >> (bits) : \ (val) << (bits)) @@ -576,12 +525,44 @@ struct fb_info { extern int fb_set_var(struct fb_info *info, struct fb_var_screeninfo *var); extern int fb_pan_display(struct fb_info *info, struct fb_var_screeninfo *var); extern int fb_blank(struct fb_info *info, int blank); + +/* + * Drawing operations where framebuffer is in I/O memory + */ + extern void cfb_fillrect(struct fb_info *info, const struct fb_fillrect *rect); extern void cfb_copyarea(struct fb_info *info, const struct fb_copyarea *area); extern void cfb_imageblit(struct fb_info *info, const struct fb_image *image); +extern ssize_t fb_io_read(struct fb_info *info, char __user *buf, + size_t count, loff_t *ppos); +extern ssize_t fb_io_write(struct fb_info *info, const char __user *buf, + size_t count, loff_t *ppos); + +/* + * Initializes struct fb_ops for framebuffers in I/O memory. + */ + +#define __FB_DEFAULT_IO_OPS_RDWR \ + .fb_read = fb_io_read, \ + .fb_write = fb_io_write + +#define __FB_DEFAULT_IO_OPS_DRAW \ + .fb_fillrect = cfb_fillrect, \ + .fb_copyarea = cfb_copyarea, \ + .fb_imageblit = cfb_imageblit + +#define __FB_DEFAULT_IO_OPS_MMAP \ + .fb_mmap = NULL /* default implementation */ + +#define FB_DEFAULT_IO_OPS \ + __FB_DEFAULT_IO_OPS_RDWR, \ + __FB_DEFAULT_IO_OPS_DRAW, \ + __FB_DEFAULT_IO_OPS_MMAP + /* * Drawing operations where framebuffer is in system RAM */ + extern void sys_fillrect(struct fb_info *info, const struct fb_fillrect *rect); extern void sys_copyarea(struct fb_info *info, const struct fb_copyarea *area); extern void sys_imageblit(struct fb_info *info, const struct fb_image *image); @@ -590,6 +571,27 @@ extern ssize_t fb_sys_read(struct fb_info *info, char __user *buf, extern ssize_t fb_sys_write(struct fb_info *info, const char __user *buf, size_t count, loff_t *ppos); +/* + * Initializes struct fb_ops for framebuffers in system memory. + */ + +#define __FB_DEFAULT_SYS_OPS_RDWR \ + .fb_read = fb_sys_read, \ + .fb_write = fb_sys_write + +#define __FB_DEFAULT_SYS_OPS_DRAW \ + .fb_fillrect = sys_fillrect, \ + .fb_copyarea = sys_copyarea, \ + .fb_imageblit = sys_imageblit + +#define __FB_DEFAULT_SYS_OPS_MMAP \ + .fb_mmap = NULL /* default implementation */ + +#define FB_DEFAULT_SYS_OPS \ + __FB_DEFAULT_SYS_OPS_RDWR, \ + __FB_DEFAULT_SYS_OPS_DRAW, \ + __FB_DEFAULT_SYS_OPS_MMAP + /* drivers/video/fbmem.c */ extern int register_framebuffer(struct fb_info *fb_info); extern void unregister_framebuffer(struct fb_info *fb_info); @@ -645,6 +647,75 @@ extern void fb_deferred_io_cleanup(struct fb_info *info); extern int fb_deferred_io_fsync(struct file *file, loff_t start, loff_t end, int datasync); +/* + * Generate callbacks for deferred I/O + */ + +#define __FB_GEN_DEFAULT_DEFERRED_OPS_RDWR(__prefix, __damage_range, __mode) \ + static ssize_t __prefix ## _defio_read(struct fb_info *info, char __user *buf, \ + size_t count, loff_t *ppos) \ + { \ + return fb_ ## __mode ## _read(info, buf, count, ppos); \ + } \ + static ssize_t __prefix ## _defio_write(struct fb_info *info, const char __user *buf, \ + size_t count, loff_t *ppos) \ + { \ + unsigned long offset = *ppos; \ + ssize_t ret = fb_ ## __mode ## _write(info, buf, count, ppos); \ + if (ret > 0) \ + __damage_range(info, offset, ret); \ + return ret; \ + } + +#define __FB_GEN_DEFAULT_DEFERRED_OPS_DRAW(__prefix, __damage_area, __mode) \ + static void __prefix ## _defio_fillrect(struct fb_info *info, \ + const struct fb_fillrect *rect) \ + { \ + __mode ## _fillrect(info, rect); \ + __damage_area(info, rect->dx, rect->dy, rect->width, rect->height); \ + } \ + static void __prefix ## _defio_copyarea(struct fb_info *info, \ + const struct fb_copyarea *area) \ + { \ + __mode ## _copyarea(info, area); \ + __damage_area(info, area->dx, area->dy, area->width, area->height); \ + } \ + static void __prefix ## _defio_imageblit(struct fb_info *info, \ + const struct fb_image *image) \ + { \ + __mode ## _imageblit(info, image); \ + __damage_area(info, image->dx, image->dy, image->width, image->height); \ + } + +#define FB_GEN_DEFAULT_DEFERRED_IO_OPS(__prefix, __damage_range, __damage_area) \ + __FB_GEN_DEFAULT_DEFERRED_OPS_RDWR(__prefix, __damage_range, io) \ + __FB_GEN_DEFAULT_DEFERRED_OPS_DRAW(__prefix, __damage_area, cfb) + +#define FB_GEN_DEFAULT_DEFERRED_SYS_OPS(__prefix, __damage_range, __damage_area) \ + __FB_GEN_DEFAULT_DEFERRED_OPS_RDWR(__prefix, __damage_range, sys) \ + __FB_GEN_DEFAULT_DEFERRED_OPS_DRAW(__prefix, __damage_area, sys) + +/* + * Initializes struct fb_ops for deferred I/O. + */ + +#define __FB_DEFAULT_DEFERRED_OPS_RDWR(__prefix) \ + .fb_read = __prefix ## _defio_read, \ + .fb_write = __prefix ## _defio_write + +#define __FB_DEFAULT_DEFERRED_OPS_DRAW(__prefix) \ + .fb_fillrect = __prefix ## _defio_fillrect, \ + .fb_copyarea = __prefix ## _defio_copyarea, \ + .fb_imageblit = __prefix ## _defio_imageblit + +#define __FB_DEFAULT_DEFERRED_OPS_MMAP(__prefix) \ + .fb_mmap = fb_deferred_io_mmap + +#define FB_DEFAULT_DEFERRED_OPS(__prefix) \ + __FB_DEFAULT_DEFERRED_OPS_RDWR(__prefix), \ + __FB_DEFAULT_DEFERRED_OPS_DRAW(__prefix), \ + __FB_DEFAULT_DEFERRED_OPS_MMAP(__prefix) + static inline bool fb_be_math(struct fb_info *info) { #ifdef CONFIG_FB_FOREIGN_ENDIAN diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index 7173d6d401eb..206371edf943 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -151,6 +151,9 @@ #define PCI_CLASS_SP_DPIO 0x1100 #define PCI_CLASS_SP_OTHER 0x1180 +#define PCI_BASE_CLASS_ACCELERATOR 0x12 +#define PCI_CLASS_ACCELERATOR_PROCESSING 0x1200 + #define PCI_CLASS_OTHERS 0xff /* Vendors and devices. Sort key: vendor first, device next. */ diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h index b6eb90df5d05..79b14828d542 100644 --- a/include/uapi/drm/amdgpu_drm.h +++ b/include/uapi/drm/amdgpu_drm.h @@ -245,6 +245,8 @@ union drm_amdgpu_bo_list { /* indicate some errors are detected by RAS */ #define AMDGPU_CTX_QUERY2_FLAGS_RAS_CE (1<<3) #define AMDGPU_CTX_QUERY2_FLAGS_RAS_UE (1<<4) +/* indicate that the reset hasn't completed yet */ +#define AMDGPU_CTX_QUERY2_FLAGS_RESET_IN_PROGRESS (1<<5) /* Context priority level */ #define AMDGPU_CTX_PRIORITY_UNSET -2048 @@ -592,6 +594,7 @@ struct drm_amdgpu_gem_va { #define AMDGPU_CHUNK_ID_SCHEDULED_DEPENDENCIES 0x07 #define AMDGPU_CHUNK_ID_SYNCOBJ_TIMELINE_WAIT 0x08 #define AMDGPU_CHUNK_ID_SYNCOBJ_TIMELINE_SIGNAL 0x09 +#define AMDGPU_CHUNK_ID_CP_GFX_SHADOW 0x0a struct drm_amdgpu_cs_chunk { __u32 chunk_id; @@ -708,6 +711,15 @@ struct drm_amdgpu_cs_chunk_data { }; }; +#define AMDGPU_CS_CHUNK_CP_GFX_SHADOW_FLAGS_INIT_SHADOW 0x1 + +struct drm_amdgpu_cs_chunk_cp_gfx_shadow { + __u64 shadow_va; + __u64 csa_va; + __u64 gds_va; + __u64 flags; +}; + /* * Query h/w info: Flag that this is integrated (a.h.a. fusion) GPU * @@ -876,6 +888,8 @@ struct drm_amdgpu_cs_chunk_data { #define AMDGPU_INFO_VIDEO_CAPS_DECODE 0 /* Subquery id: Encode */ #define AMDGPU_INFO_VIDEO_CAPS_ENCODE 1 +/* Query the max number of IBs per gang per submission */ +#define AMDGPU_INFO_MAX_IBS 0x22 #define AMDGPU_INFO_MMR_SE_INDEX_SHIFT 0 #define AMDGPU_INFO_MMR_SE_INDEX_MASK 0xff @@ -1126,6 +1140,14 @@ struct drm_amdgpu_info_device { __u64 mall_size; /* AKA infinity cache */ /* high 32 bits of the rb pipes mask */ __u32 enabled_rb_pipes_mask_hi; + /* shadow area size for gfx11 */ + __u32 shadow_size; + /* shadow area base virtual alignment for gfx11 */ + __u32 shadow_alignment; + /* context save area size for gfx11 */ + __u32 csa_size; + /* context save area base virtual alignment for gfx11 */ + __u32 csa_alignment; }; struct drm_amdgpu_info_hw_ip { diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h index de703c6be969..8db7fd3f743e 100644 --- a/include/uapi/drm/drm_fourcc.h +++ b/include/uapi/drm/drm_fourcc.h @@ -658,6 +658,49 @@ extern "C" { #define I915_FORMAT_MOD_4_TILED_DG2_RC_CCS_CC fourcc_mod_code(INTEL, 12) /* + * Intel Color Control Surfaces (CCS) for display ver. 14 render compression. + * + * The main surface is tile4 and at plane index 0, the CCS is linear and + * at index 1. A 64B CCS cache line corresponds to an area of 4x1 tiles in + * main surface. In other words, 4 bits in CCS map to a main surface cache + * line pair. The main surface pitch is required to be a multiple of four + * tile4 widths. + */ +#define I915_FORMAT_MOD_4_TILED_MTL_RC_CCS fourcc_mod_code(INTEL, 13) + +/* + * Intel Color Control Surfaces (CCS) for display ver. 14 media compression + * + * The main surface is tile4 and at plane index 0, the CCS is linear and + * at index 1. A 64B CCS cache line corresponds to an area of 4x1 tiles in + * main surface. In other words, 4 bits in CCS map to a main surface cache + * line pair. The main surface pitch is required to be a multiple of four + * tile4 widths. For semi-planar formats like NV12, CCS planes follow the + * Y and UV planes i.e., planes 0 and 1 are used for Y and UV surfaces, + * planes 2 and 3 for the respective CCS. + */ +#define I915_FORMAT_MOD_4_TILED_MTL_MC_CCS fourcc_mod_code(INTEL, 14) + +/* + * Intel Color Control Surface with Clear Color (CCS) for display ver. 14 render + * compression. + * + * The main surface is tile4 and is at plane index 0 whereas CCS is linear + * and at index 1. The clear color is stored at index 2, and the pitch should + * be ignored. The clear color structure is 256 bits. The first 128 bits + * represents Raw Clear Color Red, Green, Blue and Alpha color each represented + * by 32 bits. The raw clear color is consumed by the 3d engine and generates + * the converted clear color of size 64 bits. The first 32 bits store the Lower + * Converted Clear Color value and the next 32 bits store the Higher Converted + * Clear Color value when applicable. The Converted Clear Color values are + * consumed by the DE. The last 64 bits are used to store Color Discard Enable + * and Depth Clear Value Valid which are ignored by the DE. A CCS cache line + * corresponds to an area of 4x1 tiles in the main surface. The main surface + * pitch is required to be a multiple of 4 tile widths. + */ +#define I915_FORMAT_MOD_4_TILED_MTL_RC_CCS_CC fourcc_mod_code(INTEL, 15) + +/* * Tiled, NV12MT, grouped in 64 (pixels) x 32 (lines) -sized macroblocks * * Macroblocks are laid in a Z-shape, and each pixel data is following the diff --git a/include/uapi/drm/drm_mode.h b/include/uapi/drm/drm_mode.h index 46becedf5b2f..43691058d28f 100644 --- a/include/uapi/drm/drm_mode.h +++ b/include/uapi/drm/drm_mode.h @@ -834,6 +834,11 @@ struct drm_color_ctm { /* * Conversion matrix in S31.32 sign-magnitude * (not two's complement!) format. + * + * out matrix in + * |R| |0 1 2| |R| + * |G| = |3 4 5| x |G| + * |B| |6 7 8| |B| */ __u64 matrix[9]; }; diff --git a/include/uapi/drm/habanalabs_accel.h b/include/uapi/drm/habanalabs_accel.h index d9ef1b151d04..e6436f3e8ea6 100644 --- a/include/uapi/drm/habanalabs_accel.h +++ b/include/uapi/drm/habanalabs_accel.h @@ -787,18 +787,28 @@ enum hl_server_type { * The address which accessing it caused the razwi. * Razwi initiator. * Razwi cause, was it a page fault or MMU access error. + * May return 0 even though no new data is available, in that case + * timestamp will be 0. * HL_INFO_DEV_MEM_ALLOC_PAGE_SIZES - Retrieve valid page sizes for device memory allocation * HL_INFO_SECURED_ATTESTATION - Retrieve attestation report of the boot. * HL_INFO_REGISTER_EVENTFD - Register eventfd for event notifications. * HL_INFO_UNREGISTER_EVENTFD - Unregister eventfd * HL_INFO_GET_EVENTS - Retrieve the last occurred events * HL_INFO_UNDEFINED_OPCODE_EVENT - Retrieve last undefined opcode error information. + * May return 0 even though no new data is available, in that case + * timestamp will be 0. * HL_INFO_ENGINE_STATUS - Retrieve the status of all the h/w engines in the asic. * HL_INFO_PAGE_FAULT_EVENT - Retrieve parameters of captured page fault. + * May return 0 even though no new data is available, in that case + * timestamp will be 0. * HL_INFO_USER_MAPPINGS - Retrieve user mappings, captured after page fault event. * HL_INFO_FW_GENERIC_REQ - Send generic request to FW. * HL_INFO_HW_ERR_EVENT - Retrieve information on the reported HW error. + * May return 0 even though no new data is available, in that case + * timestamp will be 0. * HL_INFO_FW_ERR_EVENT - Retrieve information on the reported FW error. + * May return 0 even though no new data is available, in that case + * timestamp will be 0. */ #define HL_INFO_HW_IP_INFO 0 #define HL_INFO_HW_EVENTS 1 diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h index dba7c5a5b25e..7000e5910a1d 100644 --- a/include/uapi/drm/i915_drm.h +++ b/include/uapi/drm/i915_drm.h @@ -280,7 +280,16 @@ enum drm_i915_pmu_engine_sample { #define I915_PMU_ENGINE_SEMA(class, instance) \ __I915_PMU_ENGINE(class, instance, I915_SAMPLE_SEMA) -#define __I915_PMU_OTHER(x) (__I915_PMU_ENGINE(0xff, 0xff, 0xf) + 1 + (x)) +/* + * Top 4 bits of every non-engine counter are GT id. + */ +#define __I915_PMU_GT_SHIFT (60) + +#define ___I915_PMU_OTHER(gt, x) \ + (((__u64)__I915_PMU_ENGINE(0xff, 0xff, 0xf) + 1 + (x)) | \ + ((__u64)(gt) << __I915_PMU_GT_SHIFT)) + +#define __I915_PMU_OTHER(x) ___I915_PMU_OTHER(0, x) #define I915_PMU_ACTUAL_FREQUENCY __I915_PMU_OTHER(0) #define I915_PMU_REQUESTED_FREQUENCY __I915_PMU_OTHER(1) @@ -290,6 +299,12 @@ enum drm_i915_pmu_engine_sample { #define I915_PMU_LAST /* Deprecated - do not use */ I915_PMU_RC6_RESIDENCY +#define __I915_PMU_ACTUAL_FREQUENCY(gt) ___I915_PMU_OTHER(gt, 0) +#define __I915_PMU_REQUESTED_FREQUENCY(gt) ___I915_PMU_OTHER(gt, 1) +#define __I915_PMU_INTERRUPTS(gt) ___I915_PMU_OTHER(gt, 2) +#define __I915_PMU_RC6_RESIDENCY(gt) ___I915_PMU_OTHER(gt, 3) +#define __I915_PMU_SOFTWARE_GT_AWAKE_TIME(gt) ___I915_PMU_OTHER(gt, 4) + /* Each region is a minimum of 16k, and there are at most 255 of them. */ #define I915_NR_TEX_REGIONS 255 /* table size 2k - maximum due to use @@ -659,7 +674,8 @@ typedef struct drm_i915_irq_wait { * If the IOCTL is successful, the returned parameter will be set to one of the * following values: * * 0 if HuC firmware load is not complete, - * * 1 if HuC firmware is authenticated and running. + * * 1 if HuC firmware is loaded and fully authenticated, + * * 2 if HuC firmware is loaded and authenticated for clear media only */ #define I915_PARAM_HUC_STATUS 42 @@ -771,6 +787,25 @@ typedef struct drm_i915_irq_wait { */ #define I915_PARAM_OA_TIMESTAMP_FREQUENCY 57 +/* + * Query the status of PXP support in i915. + * + * The query can fail in the following scenarios with the listed error codes: + * -ENODEV = PXP support is not available on the GPU device or in the + * kernel due to missing component drivers or kernel configs. + * + * If the IOCTL is successful, the returned parameter will be set to one of + * the following values: + * 1 = PXP feature is supported and is ready for use. + * 2 = PXP feature is supported but should be ready soon (pending + * initialization of non-i915 system dependencies). + * + * NOTE: When param is supported (positive return values), user space should + * still refer to the GEM PXP context-creation UAPI header specs to be + * aware of possible failure due to system state machine at the time. + */ +#define I915_PARAM_PXP_STATUS 58 + /* Must be kept compact -- no holes and well documented */ /** @@ -2096,6 +2131,21 @@ struct drm_i915_gem_context_param { * * -ENODEV: feature not available * -EPERM: trying to mark a recoverable or not bannable context as protected + * -ENXIO: A dependency such as a component driver or firmware is not yet + * loaded so user space may need to attempt again. Depending on the + * device, this error may be reported if protected context creation is + * attempted very early after kernel start because the internal timeout + * waiting for such dependencies is not guaranteed to be larger than + * required (numbers differ depending on system and kernel config): + * - ADL/RPL: dependencies may take up to 3 seconds from kernel start + * while context creation internal timeout is 250 milisecs + * - MTL: dependencies may take up to 8 seconds from kernel start + * while context creation internal timeout is 250 milisecs + * NOTE: such dependencies happen once, so a subsequent call to create a + * protected context after a prior successful call will not experience + * such timeouts and will not return -ENXIO (unless the driver is reloaded, + * or, depending on the device, resumes from a suspended state). + * -EIO: The firmware did not succeed in creating the protected context. */ #define I915_CONTEXT_PARAM_PROTECTED_CONTENT 0xd /* Must be kept compact -- no holes and well documented */ @@ -3630,9 +3680,13 @@ struct drm_i915_gem_create_ext { * * For I915_GEM_CREATE_EXT_PROTECTED_CONTENT usage see * struct drm_i915_gem_create_ext_protected_content. + * + * For I915_GEM_CREATE_EXT_SET_PAT usage see + * struct drm_i915_gem_create_ext_set_pat. */ #define I915_GEM_CREATE_EXT_MEMORY_REGIONS 0 #define I915_GEM_CREATE_EXT_PROTECTED_CONTENT 1 +#define I915_GEM_CREATE_EXT_SET_PAT 2 __u64 extensions; }; @@ -3747,6 +3801,43 @@ struct drm_i915_gem_create_ext_protected_content { __u32 flags; }; +/** + * struct drm_i915_gem_create_ext_set_pat - The + * I915_GEM_CREATE_EXT_SET_PAT extension. + * + * If this extension is provided, the specified caching policy (PAT index) is + * applied to the buffer object. + * + * Below is an example on how to create an object with specific caching policy: + * + * .. code-block:: C + * + * struct drm_i915_gem_create_ext_set_pat set_pat_ext = { + * .base = { .name = I915_GEM_CREATE_EXT_SET_PAT }, + * .pat_index = 0, + * }; + * struct drm_i915_gem_create_ext create_ext = { + * .size = PAGE_SIZE, + * .extensions = (uintptr_t)&set_pat_ext, + * }; + * + * int err = ioctl(fd, DRM_IOCTL_I915_GEM_CREATE_EXT, &create_ext); + * if (err) ... + */ +struct drm_i915_gem_create_ext_set_pat { + /** @base: Extension link. See struct i915_user_extension. */ + struct i915_user_extension base; + /** + * @pat_index: PAT index to be set + * PAT index is a bit field in Page Table Entry to control caching + * behaviors for GPU accesses. The definition of PAT index is + * platform dependent and can be found in hardware specifications, + */ + __u32 pat_index; + /** @rsvd: reserved for future use */ + __u32 rsvd; +}; + /* ID of the protected content session managed by i915 when PXP is active */ #define I915_PROTECTED_CONTENT_DEFAULT_SESSION 0xf diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h index 2da5c3ad71bd..eeb2fdcbdcb7 100644 --- a/include/uapi/linux/kfd_ioctl.h +++ b/include/uapi/linux/kfd_ioctl.h @@ -38,9 +38,11 @@ * - 1.10 - Add SMI profiler event log * - 1.11 - Add unified memory for ctx save/restore area * - 1.12 - Add DMA buf export ioctl + * - 1.13 - Add debugger API + * - 1.14 - Update kfd_event_data */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 12 +#define KFD_IOCTL_MINOR_VERSION 14 struct kfd_ioctl_get_version_args { __u32 major_version; /* from KFD */ @@ -110,6 +112,32 @@ struct kfd_ioctl_get_available_memory_args { __u32 pad; }; +struct kfd_dbg_device_info_entry { + __u64 exception_status; + __u64 lds_base; + __u64 lds_limit; + __u64 scratch_base; + __u64 scratch_limit; + __u64 gpuvm_base; + __u64 gpuvm_limit; + __u32 gpu_id; + __u32 location_id; + __u32 vendor_id; + __u32 device_id; + __u32 revision_id; + __u32 subsystem_vendor_id; + __u32 subsystem_device_id; + __u32 fw_version; + __u32 gfx_target_version; + __u32 simd_count; + __u32 max_waves_per_simd; + __u32 array_count; + __u32 simd_arrays_per_engine; + __u32 num_xcc; + __u32 capability; + __u32 debug_prop; +}; + /* For kfd_ioctl_set_memory_policy_args.default_policy and alternate_policy */ #define KFD_IOC_CACHE_POLICY_COHERENT 0 #define KFD_IOC_CACHE_POLICY_NONCOHERENT 1 @@ -293,12 +321,20 @@ struct kfd_hsa_hw_exception_data { __u32 gpu_id; }; +/* hsa signal event data */ +struct kfd_hsa_signal_event_data { + __u64 last_event_age; /* to and from KFD */ +}; + /* Event data */ struct kfd_event_data { union { + /* From KFD */ struct kfd_hsa_memory_exception_data memory_exception_data; struct kfd_hsa_hw_exception_data hw_exception_data; - }; /* From KFD */ + /* To and From KFD */ + struct kfd_hsa_signal_event_data signal_event_data; + }; __u64 kfd_event_data_ext; /* pointer to an extension structure for future exception types */ __u32 event_id; /* to KFD */ @@ -773,6 +809,640 @@ struct kfd_ioctl_set_xnack_mode_args { __s32 xnack_enabled; }; +/* Wave launch override modes */ +enum kfd_dbg_trap_override_mode { + KFD_DBG_TRAP_OVERRIDE_OR = 0, + KFD_DBG_TRAP_OVERRIDE_REPLACE = 1 +}; + +/* Wave launch overrides */ +enum kfd_dbg_trap_mask { + KFD_DBG_TRAP_MASK_FP_INVALID = 1, + KFD_DBG_TRAP_MASK_FP_INPUT_DENORMAL = 2, + KFD_DBG_TRAP_MASK_FP_DIVIDE_BY_ZERO = 4, + KFD_DBG_TRAP_MASK_FP_OVERFLOW = 8, + KFD_DBG_TRAP_MASK_FP_UNDERFLOW = 16, + KFD_DBG_TRAP_MASK_FP_INEXACT = 32, + KFD_DBG_TRAP_MASK_INT_DIVIDE_BY_ZERO = 64, + KFD_DBG_TRAP_MASK_DBG_ADDRESS_WATCH = 128, + KFD_DBG_TRAP_MASK_DBG_MEMORY_VIOLATION = 256, + KFD_DBG_TRAP_MASK_TRAP_ON_WAVE_START = (1 << 30), + KFD_DBG_TRAP_MASK_TRAP_ON_WAVE_END = (1 << 31) +}; + +/* Wave launch modes */ +enum kfd_dbg_trap_wave_launch_mode { + KFD_DBG_TRAP_WAVE_LAUNCH_MODE_NORMAL = 0, + KFD_DBG_TRAP_WAVE_LAUNCH_MODE_HALT = 1, + KFD_DBG_TRAP_WAVE_LAUNCH_MODE_DEBUG = 3 +}; + +/* Address watch modes */ +enum kfd_dbg_trap_address_watch_mode { + KFD_DBG_TRAP_ADDRESS_WATCH_MODE_READ = 0, + KFD_DBG_TRAP_ADDRESS_WATCH_MODE_NONREAD = 1, + KFD_DBG_TRAP_ADDRESS_WATCH_MODE_ATOMIC = 2, + KFD_DBG_TRAP_ADDRESS_WATCH_MODE_ALL = 3 +}; + +/* Additional wave settings */ +enum kfd_dbg_trap_flags { + KFD_DBG_TRAP_FLAG_SINGLE_MEM_OP = 1, +}; + +/* Trap exceptions */ +enum kfd_dbg_trap_exception_code { + EC_NONE = 0, + /* per queue */ + EC_QUEUE_WAVE_ABORT = 1, + EC_QUEUE_WAVE_TRAP = 2, + EC_QUEUE_WAVE_MATH_ERROR = 3, + EC_QUEUE_WAVE_ILLEGAL_INSTRUCTION = 4, + EC_QUEUE_WAVE_MEMORY_VIOLATION = 5, + EC_QUEUE_WAVE_APERTURE_VIOLATION = 6, + EC_QUEUE_PACKET_DISPATCH_DIM_INVALID = 16, + EC_QUEUE_PACKET_DISPATCH_GROUP_SEGMENT_SIZE_INVALID = 17, + EC_QUEUE_PACKET_DISPATCH_CODE_INVALID = 18, + EC_QUEUE_PACKET_RESERVED = 19, + EC_QUEUE_PACKET_UNSUPPORTED = 20, + EC_QUEUE_PACKET_DISPATCH_WORK_GROUP_SIZE_INVALID = 21, + EC_QUEUE_PACKET_DISPATCH_REGISTER_INVALID = 22, + EC_QUEUE_PACKET_VENDOR_UNSUPPORTED = 23, + EC_QUEUE_PREEMPTION_ERROR = 30, + EC_QUEUE_NEW = 31, + /* per device */ + EC_DEVICE_QUEUE_DELETE = 32, + EC_DEVICE_MEMORY_VIOLATION = 33, + EC_DEVICE_RAS_ERROR = 34, + EC_DEVICE_FATAL_HALT = 35, + EC_DEVICE_NEW = 36, + /* per process */ + EC_PROCESS_RUNTIME = 48, + EC_PROCESS_DEVICE_REMOVE = 49, + EC_MAX +}; + +/* Mask generated by ecode in kfd_dbg_trap_exception_code */ +#define KFD_EC_MASK(ecode) (1ULL << (ecode - 1)) + +/* Masks for exception code type checks below */ +#define KFD_EC_MASK_QUEUE (KFD_EC_MASK(EC_QUEUE_WAVE_ABORT) | \ + KFD_EC_MASK(EC_QUEUE_WAVE_TRAP) | \ + KFD_EC_MASK(EC_QUEUE_WAVE_MATH_ERROR) | \ + KFD_EC_MASK(EC_QUEUE_WAVE_ILLEGAL_INSTRUCTION) | \ + KFD_EC_MASK(EC_QUEUE_WAVE_MEMORY_VIOLATION) | \ + KFD_EC_MASK(EC_QUEUE_WAVE_APERTURE_VIOLATION) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_DISPATCH_DIM_INVALID) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_DISPATCH_GROUP_SEGMENT_SIZE_INVALID) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_DISPATCH_CODE_INVALID) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_RESERVED) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_UNSUPPORTED) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_DISPATCH_WORK_GROUP_SIZE_INVALID) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_DISPATCH_REGISTER_INVALID) | \ + KFD_EC_MASK(EC_QUEUE_PACKET_VENDOR_UNSUPPORTED) | \ + KFD_EC_MASK(EC_QUEUE_PREEMPTION_ERROR) | \ + KFD_EC_MASK(EC_QUEUE_NEW)) +#define KFD_EC_MASK_DEVICE (KFD_EC_MASK(EC_DEVICE_QUEUE_DELETE) | \ + KFD_EC_MASK(EC_DEVICE_RAS_ERROR) | \ + KFD_EC_MASK(EC_DEVICE_FATAL_HALT) | \ + KFD_EC_MASK(EC_DEVICE_MEMORY_VIOLATION) | \ + KFD_EC_MASK(EC_DEVICE_NEW)) +#define KFD_EC_MASK_PROCESS (KFD_EC_MASK(EC_PROCESS_RUNTIME) | \ + KFD_EC_MASK(EC_PROCESS_DEVICE_REMOVE)) + +/* Checks for exception code types for KFD search */ +#define KFD_DBG_EC_TYPE_IS_QUEUE(ecode) \ + (!!(KFD_EC_MASK(ecode) & KFD_EC_MASK_QUEUE)) +#define KFD_DBG_EC_TYPE_IS_DEVICE(ecode) \ + (!!(KFD_EC_MASK(ecode) & KFD_EC_MASK_DEVICE)) +#define KFD_DBG_EC_TYPE_IS_PROCESS(ecode) \ + (!!(KFD_EC_MASK(ecode) & KFD_EC_MASK_PROCESS)) + + +/* Runtime enable states */ +enum kfd_dbg_runtime_state { + DEBUG_RUNTIME_STATE_DISABLED = 0, + DEBUG_RUNTIME_STATE_ENABLED = 1, + DEBUG_RUNTIME_STATE_ENABLED_BUSY = 2, + DEBUG_RUNTIME_STATE_ENABLED_ERROR = 3 +}; + +/* Runtime enable status */ +struct kfd_runtime_info { + __u64 r_debug; + __u32 runtime_state; + __u32 ttmp_setup; +}; + +/* Enable modes for runtime enable */ +#define KFD_RUNTIME_ENABLE_MODE_ENABLE_MASK 1 +#define KFD_RUNTIME_ENABLE_MODE_TTMP_SAVE_MASK 2 + +/** + * kfd_ioctl_runtime_enable_args - Arguments for runtime enable + * + * Coordinates debug exception signalling and debug device enablement with runtime. + * + * @r_debug - pointer to user struct for sharing information between ROCr and the debuggger + * @mode_mask - mask to set mode + * KFD_RUNTIME_ENABLE_MODE_ENABLE_MASK - enable runtime for debugging, otherwise disable + * KFD_RUNTIME_ENABLE_MODE_TTMP_SAVE_MASK - enable trap temporary setup (ignore on disable) + * @capabilities_mask - mask to notify runtime on what KFD supports + * + * Return - 0 on SUCCESS. + * - EBUSY if runtime enable call already pending. + * - EEXIST if user queues already active prior to call. + * If process is debug enabled, runtime enable will enable debug devices and + * wait for debugger process to send runtime exception EC_PROCESS_RUNTIME + * to unblock - see kfd_ioctl_dbg_trap_args. + * + */ +struct kfd_ioctl_runtime_enable_args { + __u64 r_debug; + __u32 mode_mask; + __u32 capabilities_mask; +}; + +/* Queue information */ +struct kfd_queue_snapshot_entry { + __u64 exception_status; + __u64 ring_base_address; + __u64 write_pointer_address; + __u64 read_pointer_address; + __u64 ctx_save_restore_address; + __u32 queue_id; + __u32 gpu_id; + __u32 ring_size; + __u32 queue_type; + __u32 ctx_save_restore_area_size; + __u32 reserved; +}; + +/* Queue status return for suspend/resume */ +#define KFD_DBG_QUEUE_ERROR_BIT 30 +#define KFD_DBG_QUEUE_INVALID_BIT 31 +#define KFD_DBG_QUEUE_ERROR_MASK (1 << KFD_DBG_QUEUE_ERROR_BIT) +#define KFD_DBG_QUEUE_INVALID_MASK (1 << KFD_DBG_QUEUE_INVALID_BIT) + +/* Context save area header information */ +struct kfd_context_save_area_header { + struct { + __u32 control_stack_offset; + __u32 control_stack_size; + __u32 wave_state_offset; + __u32 wave_state_size; + } wave_state; + __u32 debug_offset; + __u32 debug_size; + __u64 err_payload_addr; + __u32 err_event_id; + __u32 reserved1; +}; + +/* + * Debug operations + * + * For specifics on usage and return values, see documentation per operation + * below. Otherwise, generic error returns apply: + * - ESRCH if the process to debug does not exist. + * + * - EINVAL (with KFD_IOC_DBG_TRAP_ENABLE exempt) if operation + * KFD_IOC_DBG_TRAP_ENABLE has not succeeded prior. + * Also returns this error if GPU hardware scheduling is not supported. + * + * - EPERM (with KFD_IOC_DBG_TRAP_DISABLE exempt) if target process is not + * PTRACE_ATTACHED. KFD_IOC_DBG_TRAP_DISABLE is exempt to allow + * clean up of debug mode as long as process is debug enabled. + * + * - EACCES if any DBG_HW_OP (debug hardware operation) is requested when + * AMDKFD_IOC_RUNTIME_ENABLE has not succeeded prior. + * + * - ENODEV if any GPU does not support debugging on a DBG_HW_OP call. + * + * - Other errors may be returned when a DBG_HW_OP occurs while the GPU + * is in a fatal state. + * + */ +enum kfd_dbg_trap_operations { + KFD_IOC_DBG_TRAP_ENABLE = 0, + KFD_IOC_DBG_TRAP_DISABLE = 1, + KFD_IOC_DBG_TRAP_SEND_RUNTIME_EVENT = 2, + KFD_IOC_DBG_TRAP_SET_EXCEPTIONS_ENABLED = 3, + KFD_IOC_DBG_TRAP_SET_WAVE_LAUNCH_OVERRIDE = 4, /* DBG_HW_OP */ + KFD_IOC_DBG_TRAP_SET_WAVE_LAUNCH_MODE = 5, /* DBG_HW_OP */ + KFD_IOC_DBG_TRAP_SUSPEND_QUEUES = 6, /* DBG_HW_OP */ + KFD_IOC_DBG_TRAP_RESUME_QUEUES = 7, /* DBG_HW_OP */ + KFD_IOC_DBG_TRAP_SET_NODE_ADDRESS_WATCH = 8, /* DBG_HW_OP */ + KFD_IOC_DBG_TRAP_CLEAR_NODE_ADDRESS_WATCH = 9, /* DBG_HW_OP */ + KFD_IOC_DBG_TRAP_SET_FLAGS = 10, + KFD_IOC_DBG_TRAP_QUERY_DEBUG_EVENT = 11, + KFD_IOC_DBG_TRAP_QUERY_EXCEPTION_INFO = 12, + KFD_IOC_DBG_TRAP_GET_QUEUE_SNAPSHOT = 13, + KFD_IOC_DBG_TRAP_GET_DEVICE_SNAPSHOT = 14 +}; + +/** + * kfd_ioctl_dbg_trap_enable_args + * + * Arguments for KFD_IOC_DBG_TRAP_ENABLE. + * + * Enables debug session for target process. Call @op KFD_IOC_DBG_TRAP_DISABLE in + * kfd_ioctl_dbg_trap_args to disable debug session. + * + * @exception_mask (IN) - exceptions to raise to the debugger + * @rinfo_ptr (IN) - pointer to runtime info buffer (see kfd_runtime_info) + * @rinfo_size (IN/OUT) - size of runtime info buffer in bytes + * @dbg_fd (IN) - fd the KFD will nofify the debugger with of raised + * exceptions set in exception_mask. + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * Copies KFD saved kfd_runtime_info to @rinfo_ptr on enable. + * Size of kfd_runtime saved by the KFD returned to @rinfo_size. + * - EBADF if KFD cannot get a reference to dbg_fd. + * - EFAULT if KFD cannot copy runtime info to rinfo_ptr. + * - EINVAL if target process is already debug enabled. + * + */ +struct kfd_ioctl_dbg_trap_enable_args { + __u64 exception_mask; + __u64 rinfo_ptr; + __u32 rinfo_size; + __u32 dbg_fd; +}; + +/** + * kfd_ioctl_dbg_trap_send_runtime_event_args + * + * + * Arguments for KFD_IOC_DBG_TRAP_SEND_RUNTIME_EVENT. + * Raises exceptions to runtime. + * + * @exception_mask (IN) - exceptions to raise to runtime + * @gpu_id (IN) - target device id + * @queue_id (IN) - target queue id + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * - ENODEV if gpu_id not found. + * If exception_mask contains EC_PROCESS_RUNTIME, unblocks pending + * AMDKFD_IOC_RUNTIME_ENABLE call - see kfd_ioctl_runtime_enable_args. + * All other exceptions are raised to runtime through err_payload_addr. + * See kfd_context_save_area_header. + */ +struct kfd_ioctl_dbg_trap_send_runtime_event_args { + __u64 exception_mask; + __u32 gpu_id; + __u32 queue_id; +}; + +/** + * kfd_ioctl_dbg_trap_set_exceptions_enabled_args + * + * Arguments for KFD_IOC_SET_EXCEPTIONS_ENABLED + * Set new exceptions to be raised to the debugger. + * + * @exception_mask (IN) - new exceptions to raise the debugger + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + */ +struct kfd_ioctl_dbg_trap_set_exceptions_enabled_args { + __u64 exception_mask; +}; + +/** + * kfd_ioctl_dbg_trap_set_wave_launch_override_args + * + * Arguments for KFD_IOC_DBG_TRAP_SET_WAVE_LAUNCH_OVERRIDE + * Enable HW exceptions to raise trap. + * + * @override_mode (IN) - see kfd_dbg_trap_override_mode + * @enable_mask (IN/OUT) - reference kfd_dbg_trap_mask. + * IN is the override modes requested to be enabled. + * OUT is referenced in Return below. + * @support_request_mask (IN/OUT) - reference kfd_dbg_trap_mask. + * IN is the override modes requested for support check. + * OUT is referenced in Return below. + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * Previous enablement is returned in @enable_mask. + * Actual override support is returned in @support_request_mask. + * - EINVAL if override mode is not supported. + * - EACCES if trap support requested is not actually supported. + * i.e. enable_mask (IN) is not a subset of support_request_mask (OUT). + * Otherwise it is considered a generic error (see kfd_dbg_trap_operations). + */ +struct kfd_ioctl_dbg_trap_set_wave_launch_override_args { + __u32 override_mode; + __u32 enable_mask; + __u32 support_request_mask; + __u32 pad; +}; + +/** + * kfd_ioctl_dbg_trap_set_wave_launch_mode_args + * + * Arguments for KFD_IOC_DBG_TRAP_SET_WAVE_LAUNCH_MODE + * Set wave launch mode. + * + * @mode (IN) - see kfd_dbg_trap_wave_launch_mode + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + */ +struct kfd_ioctl_dbg_trap_set_wave_launch_mode_args { + __u32 launch_mode; + __u32 pad; +}; + +/** + * kfd_ioctl_dbg_trap_suspend_queues_ags + * + * Arguments for KFD_IOC_DBG_TRAP_SUSPEND_QUEUES + * Suspend queues. + * + * @exception_mask (IN) - raised exceptions to clear + * @queue_array_ptr (IN) - pointer to array of queue ids (u32 per queue id) + * to suspend + * @num_queues (IN) - number of queues to suspend in @queue_array_ptr + * @grace_period (IN) - wave time allowance before preemption + * per 1K GPU clock cycle unit + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Destruction of a suspended queue is blocked until the queue is + * resumed. This allows the debugger to access queue information and + * the its context save area without running into a race condition on + * queue destruction. + * Automatically copies per queue context save area header information + * into the save area base + * (see kfd_queue_snapshot_entry and kfd_context_save_area_header). + * + * Return - Number of queues suspended on SUCCESS. + * . KFD_DBG_QUEUE_ERROR_MASK and KFD_DBG_QUEUE_INVALID_MASK masked + * for each queue id in @queue_array_ptr array reports unsuccessful + * suspend reason. + * KFD_DBG_QUEUE_ERROR_MASK = HW failure. + * KFD_DBG_QUEUE_INVALID_MASK = queue does not exist, is new or + * is being destroyed. + */ +struct kfd_ioctl_dbg_trap_suspend_queues_args { + __u64 exception_mask; + __u64 queue_array_ptr; + __u32 num_queues; + __u32 grace_period; +}; + +/** + * kfd_ioctl_dbg_trap_resume_queues_args + * + * Arguments for KFD_IOC_DBG_TRAP_RESUME_QUEUES + * Resume queues. + * + * @queue_array_ptr (IN) - pointer to array of queue ids (u32 per queue id) + * to resume + * @num_queues (IN) - number of queues to resume in @queue_array_ptr + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - Number of queues resumed on SUCCESS. + * KFD_DBG_QUEUE_ERROR_MASK and KFD_DBG_QUEUE_INVALID_MASK mask + * for each queue id in @queue_array_ptr array reports unsuccessful + * resume reason. + * KFD_DBG_QUEUE_ERROR_MASK = HW failure. + * KFD_DBG_QUEUE_INVALID_MASK = queue does not exist. + */ +struct kfd_ioctl_dbg_trap_resume_queues_args { + __u64 queue_array_ptr; + __u32 num_queues; + __u32 pad; +}; + +/** + * kfd_ioctl_dbg_trap_set_node_address_watch_args + * + * Arguments for KFD_IOC_DBG_TRAP_SET_NODE_ADDRESS_WATCH + * Sets address watch for device. + * + * @address (IN) - watch address to set + * @mode (IN) - see kfd_dbg_trap_address_watch_mode + * @mask (IN) - watch address mask + * @gpu_id (IN) - target gpu to set watch point + * @id (OUT) - watch id allocated + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * Allocated watch ID returned to @id. + * - ENODEV if gpu_id not found. + * - ENOMEM if watch IDs can be allocated + */ +struct kfd_ioctl_dbg_trap_set_node_address_watch_args { + __u64 address; + __u32 mode; + __u32 mask; + __u32 gpu_id; + __u32 id; +}; + +/** + * kfd_ioctl_dbg_trap_clear_node_address_watch_args + * + * Arguments for KFD_IOC_DBG_TRAP_CLEAR_NODE_ADDRESS_WATCH + * Clear address watch for device. + * + * @gpu_id (IN) - target device to clear watch point + * @id (IN) - allocated watch id to clear + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * - ENODEV if gpu_id not found. + * - EINVAL if watch ID has not been allocated. + */ +struct kfd_ioctl_dbg_trap_clear_node_address_watch_args { + __u32 gpu_id; + __u32 id; +}; + +/** + * kfd_ioctl_dbg_trap_set_flags_args + * + * Arguments for KFD_IOC_DBG_TRAP_SET_FLAGS + * Sets flags for wave behaviour. + * + * @flags (IN/OUT) - IN = flags to enable, OUT = flags previously enabled + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * - EACCESS if any debug device does not allow flag options. + */ +struct kfd_ioctl_dbg_trap_set_flags_args { + __u32 flags; + __u32 pad; +}; + +/** + * kfd_ioctl_dbg_trap_query_debug_event_args + * + * Arguments for KFD_IOC_DBG_TRAP_QUERY_DEBUG_EVENT + * + * Find one or more raised exceptions. This function can return multiple + * exceptions from a single queue or a single device with one call. To find + * all raised exceptions, this function must be called repeatedly until it + * returns -EAGAIN. Returned exceptions can optionally be cleared by + * setting the corresponding bit in the @exception_mask input parameter. + * However, clearing an exception prevents retrieving further information + * about it with KFD_IOC_DBG_TRAP_QUERY_EXCEPTION_INFO. + * + * @exception_mask (IN/OUT) - exception to clear (IN) and raised (OUT) + * @gpu_id (OUT) - gpu id of exceptions raised + * @queue_id (OUT) - queue id of exceptions raised + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on raised exception found + * Raised exceptions found are returned in @exception mask + * with reported source id returned in @gpu_id or @queue_id. + * - EAGAIN if no raised exception has been found + */ +struct kfd_ioctl_dbg_trap_query_debug_event_args { + __u64 exception_mask; + __u32 gpu_id; + __u32 queue_id; +}; + +/** + * kfd_ioctl_dbg_trap_query_exception_info_args + * + * Arguments KFD_IOC_DBG_TRAP_QUERY_EXCEPTION_INFO + * Get additional info on raised exception. + * + * @info_ptr (IN) - pointer to exception info buffer to copy to + * @info_size (IN/OUT) - exception info buffer size (bytes) + * @source_id (IN) - target gpu or queue id + * @exception_code (IN) - target exception + * @clear_exception (IN) - clear raised @exception_code exception + * (0 = false, 1 = true) + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * If @exception_code is EC_DEVICE_MEMORY_VIOLATION, copy @info_size(OUT) + * bytes of memory exception data to @info_ptr. + * If @exception_code is EC_PROCESS_RUNTIME, copy saved + * kfd_runtime_info to @info_ptr. + * Actual required @info_ptr size (bytes) is returned in @info_size. + */ +struct kfd_ioctl_dbg_trap_query_exception_info_args { + __u64 info_ptr; + __u32 info_size; + __u32 source_id; + __u32 exception_code; + __u32 clear_exception; +}; + +/** + * kfd_ioctl_dbg_trap_get_queue_snapshot_args + * + * Arguments KFD_IOC_DBG_TRAP_GET_QUEUE_SNAPSHOT + * Get queue information. + * + * @exception_mask (IN) - exceptions raised to clear + * @snapshot_buf_ptr (IN) - queue snapshot entry buffer (see kfd_queue_snapshot_entry) + * @num_queues (IN/OUT) - number of queue snapshot entries + * The debugger specifies the size of the array allocated in @num_queues. + * KFD returns the number of queues that actually existed. If this is + * larger than the size specified by the debugger, KFD will not overflow + * the array allocated by the debugger. + * + * @entry_size (IN/OUT) - size per entry in bytes + * The debugger specifies sizeof(struct kfd_queue_snapshot_entry) in + * @entry_size. KFD returns the number of bytes actually populated per + * entry. The debugger should use the KFD_IOCTL_MINOR_VERSION to determine, + * which fields in struct kfd_queue_snapshot_entry are valid. This allows + * growing the ABI in a backwards compatible manner. + * Note that entry_size(IN) should still be used to stride the snapshot buffer in the + * event that it's larger than actual kfd_queue_snapshot_entry. + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * Copies @num_queues(IN) queue snapshot entries of size @entry_size(IN) + * into @snapshot_buf_ptr if @num_queues(IN) > 0. + * Otherwise return @num_queues(OUT) queue snapshot entries that exist. + */ +struct kfd_ioctl_dbg_trap_queue_snapshot_args { + __u64 exception_mask; + __u64 snapshot_buf_ptr; + __u32 num_queues; + __u32 entry_size; +}; + +/** + * kfd_ioctl_dbg_trap_get_device_snapshot_args + * + * Arguments for KFD_IOC_DBG_TRAP_GET_DEVICE_SNAPSHOT + * Get device information. + * + * @exception_mask (IN) - exceptions raised to clear + * @snapshot_buf_ptr (IN) - pointer to snapshot buffer (see kfd_dbg_device_info_entry) + * @num_devices (IN/OUT) - number of debug devices to snapshot + * The debugger specifies the size of the array allocated in @num_devices. + * KFD returns the number of devices that actually existed. If this is + * larger than the size specified by the debugger, KFD will not overflow + * the array allocated by the debugger. + * + * @entry_size (IN/OUT) - size per entry in bytes + * The debugger specifies sizeof(struct kfd_dbg_device_info_entry) in + * @entry_size. KFD returns the number of bytes actually populated. The + * debugger should use KFD_IOCTL_MINOR_VERSION to determine, which fields + * in struct kfd_dbg_device_info_entry are valid. This allows growing the + * ABI in a backwards compatible manner. + * Note that entry_size(IN) should still be used to stride the snapshot buffer in the + * event that it's larger than actual kfd_dbg_device_info_entry. + * + * Generic errors apply (see kfd_dbg_trap_operations). + * Return - 0 on SUCCESS. + * Copies @num_devices(IN) device snapshot entries of size @entry_size(IN) + * into @snapshot_buf_ptr if @num_devices(IN) > 0. + * Otherwise return @num_devices(OUT) queue snapshot entries that exist. + */ +struct kfd_ioctl_dbg_trap_device_snapshot_args { + __u64 exception_mask; + __u64 snapshot_buf_ptr; + __u32 num_devices; + __u32 entry_size; +}; + +/** + * kfd_ioctl_dbg_trap_args + * + * Arguments to debug target process. + * + * @pid - target process to debug + * @op - debug operation (see kfd_dbg_trap_operations) + * + * @op determines which union struct args to use. + * Refer to kern docs for each kfd_ioctl_dbg_trap_*_args struct. + */ +struct kfd_ioctl_dbg_trap_args { + __u32 pid; + __u32 op; + + union { + struct kfd_ioctl_dbg_trap_enable_args enable; + struct kfd_ioctl_dbg_trap_send_runtime_event_args send_runtime_event; + struct kfd_ioctl_dbg_trap_set_exceptions_enabled_args set_exceptions_enabled; + struct kfd_ioctl_dbg_trap_set_wave_launch_override_args launch_override; + struct kfd_ioctl_dbg_trap_set_wave_launch_mode_args launch_mode; + struct kfd_ioctl_dbg_trap_suspend_queues_args suspend_queues; + struct kfd_ioctl_dbg_trap_resume_queues_args resume_queues; + struct kfd_ioctl_dbg_trap_set_node_address_watch_args set_node_address_watch; + struct kfd_ioctl_dbg_trap_clear_node_address_watch_args clear_node_address_watch; + struct kfd_ioctl_dbg_trap_set_flags_args set_flags; + struct kfd_ioctl_dbg_trap_query_debug_event_args query_debug_event; + struct kfd_ioctl_dbg_trap_query_exception_info_args query_exception_info; + struct kfd_ioctl_dbg_trap_queue_snapshot_args queue_snapshot; + struct kfd_ioctl_dbg_trap_device_snapshot_args device_snapshot; + }; +}; + #define AMDKFD_IOCTL_BASE 'K' #define AMDKFD_IO(nr) _IO(AMDKFD_IOCTL_BASE, nr) #define AMDKFD_IOR(nr, type) _IOR(AMDKFD_IOCTL_BASE, nr, type) @@ -887,7 +1557,13 @@ struct kfd_ioctl_set_xnack_mode_args { #define AMDKFD_IOC_EXPORT_DMABUF \ AMDKFD_IOWR(0x24, struct kfd_ioctl_export_dmabuf_args) +#define AMDKFD_IOC_RUNTIME_ENABLE \ + AMDKFD_IOWR(0x25, struct kfd_ioctl_runtime_enable_args) + +#define AMDKFD_IOC_DBG_TRAP \ + AMDKFD_IOWR(0x26, struct kfd_ioctl_dbg_trap_args) + #define AMDKFD_COMMAND_START 0x01 -#define AMDKFD_COMMAND_END 0x25 +#define AMDKFD_COMMAND_END 0x27 #endif diff --git a/include/uapi/linux/kfd_sysfs.h b/include/uapi/linux/kfd_sysfs.h index 3e330f368917..a51b7331e0b4 100644 --- a/include/uapi/linux/kfd_sysfs.h +++ b/include/uapi/linux/kfd_sysfs.h @@ -43,6 +43,11 @@ #define HSA_CAP_DOORBELL_TYPE_2_0 0x2 #define HSA_CAP_AQL_QUEUE_DOUBLE_MAP 0x00004000 +#define HSA_CAP_TRAP_DEBUG_SUPPORT 0x00008000 +#define HSA_CAP_TRAP_DEBUG_WAVE_LAUNCH_TRAP_OVERRIDE_SUPPORTED 0x00010000 +#define HSA_CAP_TRAP_DEBUG_WAVE_LAUNCH_MODE_SUPPORTED 0x00020000 +#define HSA_CAP_TRAP_DEBUG_PRECISE_MEMORY_OPERATIONS_SUPPORTED 0x00040000 + /* Old buggy user mode depends on this being 0 */ #define HSA_CAP_RESERVED_WAS_SRAM_EDCSUPPORTED 0x00080000 @@ -53,8 +58,18 @@ #define HSA_CAP_SRAM_EDCSUPPORTED 0x04000000 #define HSA_CAP_SVMAPI_SUPPORTED 0x08000000 #define HSA_CAP_FLAGS_COHERENTHOSTACCESS 0x10000000 +#define HSA_CAP_TRAP_DEBUG_FIRMWARE_SUPPORTED 0x20000000 #define HSA_CAP_RESERVED 0xe00f8000 +/* debug_prop bits in node properties */ +#define HSA_DBG_WATCH_ADDR_MASK_LO_BIT_MASK 0x0000000f +#define HSA_DBG_WATCH_ADDR_MASK_LO_BIT_SHIFT 0 +#define HSA_DBG_WATCH_ADDR_MASK_HI_BIT_MASK 0x000003f0 +#define HSA_DBG_WATCH_ADDR_MASK_HI_BIT_SHIFT 4 +#define HSA_DBG_DISPATCH_INFO_ALWAYS_VALID 0x00000400 +#define HSA_DBG_WATCHPOINTS_EXCLUSIVE 0x00000800 +#define HSA_DBG_RESERVED 0xfffffffffffff000ull + /* Heap types in memory properties */ #define HSA_MEM_HEAP_TYPE_SYSTEM 0 #define HSA_MEM_HEAP_TYPE_FB_PUBLIC 1 diff --git a/include/video/sticore.h b/include/video/sticore.h new file mode 100644 index 000000000000..fbb78d7e7565 --- /dev/null +++ b/include/video/sticore.h @@ -0,0 +1,406 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef STICORE_H +#define STICORE_H + +struct fb_info; + +/* generic STI structures & functions */ + +#define MAX_STI_ROMS 4 /* max no. of ROMs which this driver handles */ + +#define STI_REGION_MAX 8 /* hardcoded STI constants */ +#define STI_DEV_NAME_LENGTH 32 +#define STI_MONITOR_MAX 256 + +#define STI_FONT_HPROMAN8 1 +#define STI_FONT_KANA8 2 + +#define ALT_CODE_TYPE_UNKNOWN 0x00 /* alt code type values */ +#define ALT_CODE_TYPE_PA_RISC_64 0x01 + +/* The latency of the STI functions cannot really be reduced by setting + * this to 0; STI doesn't seem to be designed to allow calling a different + * function (or the same function with different arguments) after a + * function exited with 1 as return value. + * + * As all of the functions below could be called from interrupt context, + * we have to spin_lock_irqsave around the do { ret = bla(); } while(ret==1) + * block. Really bad latency there. + * + * Probably the best solution to all this is have the generic code manage + * the screen buffer and a kernel thread to call STI occasionally. + * + * Luckily, the frame buffer guys have the same problem so we can just wait + * for them to fix it and steal their solution. prumpf + */ + +#include <asm/io.h> + +#define STI_WAIT 1 + +#define STI_PTR(p) ( virt_to_phys(p) ) +#define PTR_STI(p) ( phys_to_virt((unsigned long)p) ) + +#define sti_onscreen_x(sti) (sti->glob_cfg->onscreen_x) +#define sti_onscreen_y(sti) (sti->glob_cfg->onscreen_y) + +/* sti_font_xy() use the native font ROM ! */ +#define sti_font_x(sti) (PTR_STI(sti->font)->width) +#define sti_font_y(sti) (PTR_STI(sti->font)->height) + +#ifdef CONFIG_64BIT +#define STI_LOWMEM (GFP_KERNEL | GFP_DMA) +#else +#define STI_LOWMEM (GFP_KERNEL) +#endif + + +/* STI function configuration structs */ + +typedef union region { + struct { + u32 offset : 14; /* offset in 4kbyte page */ + u32 sys_only : 1; /* don't map to user space */ + u32 cache : 1; /* map to data cache */ + u32 btlb : 1; /* map to block tlb */ + u32 last : 1; /* last region in list */ + u32 length : 14; /* length in 4kbyte page */ + } region_desc; + + u32 region; /* complete region value */ +} region_t; + +#define REGION_OFFSET_TO_PHYS( rt, hpa ) \ + (((rt).region_desc.offset << 12) + (hpa)) + +struct sti_glob_cfg_ext { + u8 curr_mon; /* current monitor configured */ + u8 friendly_boot; /* in friendly boot mode */ + s16 power; /* power calculation (in Watts) */ + s32 freq_ref; /* frequency reference */ + u32 sti_mem_addr; /* pointer to global sti memory (size=sti_mem_request) */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_glob_cfg { + s32 text_planes; /* number of planes used for text */ + s16 onscreen_x; /* screen width in pixels */ + s16 onscreen_y; /* screen height in pixels */ + s16 offscreen_x; /* offset width in pixels */ + s16 offscreen_y; /* offset height in pixels */ + s16 total_x; /* frame buffer width in pixels */ + s16 total_y; /* frame buffer height in pixels */ + u32 region_ptrs[STI_REGION_MAX]; /* region pointers */ + s32 reent_lvl; /* storage for reentry level value */ + u32 save_addr; /* where to save or restore reentrant state */ + u32 ext_ptr; /* pointer to extended glob_cfg data structure */ +}; + + +/* STI init function structs */ + +struct sti_init_flags { + u32 wait : 1; /* should routine idle wait or not */ + u32 reset : 1; /* hard reset the device? */ + u32 text : 1; /* turn on text display planes? */ + u32 nontext : 1; /* turn on non-text display planes? */ + u32 clear : 1; /* clear text display planes? */ + u32 cmap_blk : 1; /* non-text planes cmap black? */ + u32 enable_be_timer : 1; /* enable bus error timer */ + u32 enable_be_int : 1; /* enable bus error timer interrupt */ + u32 no_chg_tx : 1; /* don't change text settings */ + u32 no_chg_ntx : 1; /* don't change non-text settings */ + u32 no_chg_bet : 1; /* don't change berr timer settings */ + u32 no_chg_bei : 1; /* don't change berr int settings */ + u32 init_cmap_tx : 1; /* initialize cmap for text planes */ + u32 cmt_chg : 1; /* change current monitor type */ + u32 retain_ie : 1; /* don't allow reset to clear int enables */ + u32 caller_bootrom : 1; /* set only by bootrom for each call */ + u32 caller_kernel : 1; /* set only by kernel for each call */ + u32 caller_other : 1; /* set only by non-[BR/K] caller */ + u32 pad : 14; /* pad to word boundary */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_init_inptr_ext { + u8 config_mon_type; /* configure to monitor type */ + u8 pad[1]; /* pad to word boundary */ + u16 inflight_data; /* inflight data possible on PCI */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_init_inptr { + s32 text_planes; /* number of planes to use for text */ + u32 ext_ptr; /* pointer to extended init_graph inptr data structure*/ +}; + + +struct sti_init_outptr { + s32 errno; /* error number on failure */ + s32 text_planes; /* number of planes used for text */ + u32 future_ptr; /* pointer to future data */ +}; + + + +/* STI configuration function structs */ + +struct sti_conf_flags { + u32 wait : 1; /* should routine idle wait or not */ + u32 pad : 31; /* pad to word boundary */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_conf_inptr { + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_conf_outptr_ext { + u32 crt_config[3]; /* hardware specific X11/OGL information */ + u32 crt_hdw[3]; + u32 future_ptr; +}; + +struct sti_conf_outptr { + s32 errno; /* error number on failure */ + s16 onscreen_x; /* screen width in pixels */ + s16 onscreen_y; /* screen height in pixels */ + s16 offscreen_x; /* offscreen width in pixels */ + s16 offscreen_y; /* offscreen height in pixels */ + s16 total_x; /* frame buffer width in pixels */ + s16 total_y; /* frame buffer height in pixels */ + s32 bits_per_pixel; /* bits/pixel device has configured */ + s32 bits_used; /* bits which can be accessed */ + s32 planes; /* number of fb planes in system */ + u8 dev_name[STI_DEV_NAME_LENGTH]; /* null terminated product name */ + u32 attributes; /* flags denoting attributes */ + u32 ext_ptr; /* pointer to future data */ +}; + +struct sti_rom { + u8 type[4]; + u8 res004; + u8 num_mons; + u8 revno[2]; + u32 graphics_id[2]; + + u32 font_start; + u32 statesize; + u32 last_addr; + u32 region_list; + + u16 reentsize; + u16 maxtime; + u32 mon_tbl_addr; + u32 user_data_addr; + u32 sti_mem_req; + + u32 user_data_size; + u16 power; + u8 bus_support; + u8 ext_bus_support; + u8 alt_code_type; + u8 ext_dd_struct[3]; + u32 cfb_addr; + + u32 init_graph; + u32 state_mgmt; + u32 font_unpmv; + u32 block_move; + u32 self_test; + u32 excep_hdlr; + u32 inq_conf; + u32 set_cm_entry; + u32 dma_ctrl; + u8 res040[7 * 4]; + + u32 init_graph_addr; + u32 state_mgmt_addr; + u32 font_unp_addr; + u32 block_move_addr; + u32 self_test_addr; + u32 excep_hdlr_addr; + u32 inq_conf_addr; + u32 set_cm_entry_addr; + u32 image_unpack_addr; + u32 pa_risx_addrs[7]; +}; + +struct sti_rom_font { + u16 first_char; + u16 last_char; + u8 width; + u8 height; + u8 font_type; /* language type */ + u8 bytes_per_char; + u32 next_font; + u8 underline_height; + u8 underline_pos; + u8 res008[2]; +}; + +/* sticore internal font handling */ + +struct sti_cooked_font { + struct sti_rom_font *raw; /* native ptr for STI functions */ + void *raw_ptr; /* kmalloc'ed font data */ + struct sti_cooked_font *next_font; + int height, width; + int refcount; + u32 crc; +}; + +struct sti_cooked_rom { + struct sti_rom *raw; + struct sti_cooked_font *font_start; +}; + +/* STI font printing function structs */ + +struct sti_font_inptr { + u32 font_start_addr; /* address of font start */ + s16 index; /* index into font table of character */ + u8 fg_color; /* foreground color of character */ + u8 bg_color; /* background color of character */ + s16 dest_x; /* X location of character upper left */ + s16 dest_y; /* Y location of character upper left */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_font_flags { + u32 wait : 1; /* should routine idle wait or not */ + u32 non_text : 1; /* font unpack/move in non_text planes =1, text =0 */ + u32 pad : 30; /* pad to word boundary */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_font_outptr { + s32 errno; /* error number on failure */ + u32 future_ptr; /* pointer to future data */ +}; + +/* STI blockmove structs */ + +struct sti_blkmv_flags { + u32 wait : 1; /* should routine idle wait or not */ + u32 color : 1; /* change color during move? */ + u32 clear : 1; /* clear during move? */ + u32 non_text : 1; /* block move in non_text planes =1, text =0 */ + u32 pad : 28; /* pad to word boundary */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_blkmv_inptr { + u8 fg_color; /* foreground color after move */ + u8 bg_color; /* background color after move */ + s16 src_x; /* source upper left pixel x location */ + s16 src_y; /* source upper left pixel y location */ + s16 dest_x; /* dest upper left pixel x location */ + s16 dest_y; /* dest upper left pixel y location */ + s16 width; /* block width in pixels */ + s16 height; /* block height in pixels */ + u32 future_ptr; /* pointer to future data */ +}; + +struct sti_blkmv_outptr { + s32 errno; /* error number on failure */ + u32 future_ptr; /* pointer to future data */ +}; + + +/* sti_all_data is an internal struct which needs to be allocated in + * low memory (< 4GB) if STI is used with 32bit STI on a 64bit kernel */ + +struct sti_all_data { + struct sti_glob_cfg glob_cfg; + struct sti_glob_cfg_ext glob_cfg_ext; + + struct sti_conf_inptr inq_inptr; + struct sti_conf_outptr inq_outptr; /* configuration */ + struct sti_conf_outptr_ext inq_outptr_ext; + + struct sti_init_inptr_ext init_inptr_ext; + struct sti_init_inptr init_inptr; + struct sti_init_outptr init_outptr; + + struct sti_blkmv_inptr blkmv_inptr; + struct sti_blkmv_outptr blkmv_outptr; + + struct sti_font_inptr font_inptr; + struct sti_font_outptr font_outptr; + + /* leave as last entries */ + unsigned long save_addr[1024 / sizeof(unsigned long)]; + /* min 256 bytes which is STI default, max sti->sti_mem_request */ + unsigned long sti_mem_addr[256 / sizeof(unsigned long)]; + /* do not add something below here ! */ +}; + +/* internal generic STI struct */ + +struct sti_struct { + spinlock_t lock; + + /* char **mon_strings; */ + int sti_mem_request; + u32 graphics_id[2]; + + struct sti_cooked_rom *rom; + + unsigned long font_unpmv; + unsigned long block_move; + unsigned long init_graph; + unsigned long inq_conf; + + /* all following fields are initialized by the generic routines */ + int text_planes; + region_t regions[STI_REGION_MAX]; + unsigned long regions_phys[STI_REGION_MAX]; + + struct sti_glob_cfg *glob_cfg; /* points into sti_all_data */ + + int wordmode; + struct sti_cooked_font *font; /* ptr to selected font (cooked) */ + + struct pci_dev *pd; + + /* PCI data structures (pg. 17ff from sti.pdf) */ + u8 rm_entry[16]; /* pci region mapper array == pci config space offset */ + + /* pointer to the fb_info where this STI device is used */ + struct fb_info *info; + + /* pointer to all internal data */ + struct sti_all_data *sti_data; + + /* pa_path of this device */ + char pa_path[24]; +}; + + +/* sticore interface functions */ + +struct sti_struct *sti_get_rom(unsigned int index); /* 0: default sti */ +void sti_font_convert_bytemode(struct sti_struct *sti, struct sti_cooked_font *f); + + +/* sticore main function to call STI firmware */ + +int sti_call(const struct sti_struct *sti, unsigned long func, + const void *flags, void *inptr, void *outptr, + struct sti_glob_cfg *glob_cfg); + + +/* functions to call the STI ROM directly */ + +void sti_putc(struct sti_struct *sti, int c, int y, int x, + struct sti_cooked_font *font); +void sti_set(struct sti_struct *sti, int src_y, int src_x, + int height, int width, u8 color); +void sti_clear(struct sti_struct *sti, int src_y, int src_x, + int height, int width, int c, struct sti_cooked_font *font); +void sti_bmove(struct sti_struct *sti, int src_y, int src_x, + int dst_y, int dst_x, int height, int width, + struct sti_cooked_font *font); + +#endif /* STICORE_H */ |