[coreboot] Change in coreboot[master]: Asus F2A85-M: Activate IOMMU support

Paul Menzel paulepanter at users.sourceforge.net
Sat Jun 29 19:03:24 CEST 2013


Dear coreboot folks,


Am Montag, den 24.06.2013, 13:45 -0600 schrieb David Hubbard:

> To answer http://review.coreboot.org/3517 about what the vendor BIOS does
> for the IOMMU, here are my kernel logs (with the IOMMU enabled)
> 
> Please note:
> Command line: ... iommu=pt
> 
> I had to do that to work around (more like, bury my head in the sand) the
> problem with the on-board RTL8111/8168F, which trips over the IOMMU when
> utilization gets near 100% gigabit speed. I put the details on kernel
> bugzilla https://bugzilla.kernel.org/show_bug.cgi?id=55841

thank you a lot for collecting logs while Rudolf is busy. Also thank for
sending the logs with Radeon instead FGLRX. For the archive, I would
deem it a good thing to CC the list too.

Am Freitag, den 28.06.2013, 19:29 -0600 schrieb David Hubbard:

[…]

> Attached are two logs, the first one is the one you requested (with the
> parameter "iommu=pt" removed from the kernel command line). It does look
> like there's an interesting line in there for PCI device 1:0.0, the APU
> integrated GPU:
> 
> [    0.525376] AMD-Vi: Using passthrough domain for device 0000:01:00.0
> 
> The second log is with "iommu=pt" added back. In both cases, I loaded the
> radeon open source driver this time, instead of fglrx.

1. Please keep all in mind, that Rudolf tested with Linux 3.10-rc1 and
you are using 3.8.11.

2. Looking through the log, the vendor BIOS is not perfect either. ;-)

        […]
        [    0.000000] Checking aperture...
        [    0.000000] No AGP bridge found
        [    0.000000] Node 0: aperture @ 0 size 32 MB
        [    0.000000] Your BIOS doesn't leave a aperture memory hole
        [    0.000000] Please enable the IOMMU option in the BIOS setup
        [    0.000000] This costs you 64 MB of RAM
        [    0.000000] Mapping aperture over 65536 KB of RAM @ 84000000
        [    0.000000] Memory: 15783604k/18071552k available (3442k kernel code, 1869616k absent, 418332k reserved, 3061k data, 516k init)
        [    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
        […]


So back to the problem, the solution to the PAGE_FAULT with coreboot
seems “simple”. Using the vendor BIOS the graphics device (1:00.0) is
put into passthrough domain automatically while with coreboot it is not.
`amd_iommu.c` prints the message. (The line numbers are from Linux
3.10-rc7.)

        AMD-Vi: Using passthrough domain for device 0000:01:00.0

            $ nl -ba drivers/iommu/amd_iommu.c
            […]
              3028	/*
              3029	 * The function for pre-allocating protection domains.
              3030	 *
              3031	 * If the driver core informs the DMA layer if a driver grabs a device
              3032	 * we don't need to preallocate the protection domains anymore.
              3033	 * For now we have to.
              3034	 */
              3035	static void __init prealloc_protection_domains(void)
              3036	{
              3037		struct iommu_dev_data *dev_data;
              3038		struct dma_ops_domain *dma_dom;
              3039		struct pci_dev *dev = NULL;
              3040		u16 devid;
              3041	
              3042		for_each_pci_dev(dev) {
              3043	
              3044			/* Do we handle this device? */
              3045			if (!check_device(&dev->dev))
              3046				continue;
              3047	
              3048			dev_data = get_dev_data(&dev->dev);
              3049			if (!amd_iommu_force_isolation && dev_data->iommu_v2) {
              3050				/* Make sure passthrough domain is allocated */
              3051				alloc_passthrough_domain();
              3052				dev_data->passthrough = true;
              3053				attach_device(&dev->dev, pt_domain);
              3054				pr_info("AMD-Vi: Using passthrough domain for device %s\n",
              3055					dev_name(&dev->dev));
              3056			}
            […]

As no options were passed, the variable `dev_data->iommu_v2` seems to
make the difference.

        $ nl -ba drivers/iommu/amd_iommu_init.c
        […]
          1185		if (!(iommu->cap & (1 << IOMMU_CAP_IOTLB)))
          1186			amd_iommu_iotlb_sup = false;
          1187	
          1188		/* read extended feature bits */
          1189		low  = readl(iommu->mmio_base + MMIO_EXT_FEATURES);
          1190		high = readl(iommu->mmio_base + MMIO_EXT_FEATURES + 4);
          1191	
          1192		iommu->features = ((u64)high << 32) | low;
        […]
          1214		if (iommu_feature(iommu, FEATURE_GT) &&
          1215		    iommu_feature(iommu, FEATURE_PPR)) {
          1216			iommu->is_iommu_v2   = true;
          1217			amd_iommu_v2_present = true;
          1218		}
        […]

`iommu_feature()` is implemented as follows.

        $ nl -ba drivers/iommu/amd_iommu_proto.h
        […]
            72	static inline bool is_rd890_iommu(struct pci_dev *pdev)
            73	{
            74		return (pdev->vendor == PCI_VENDOR_ID_ATI) &&
            75		       (pdev->device == PCI_DEVICE_ID_RD890_IOMMU);
            76	}
            77	
            78	static inline bool iommu_feature(struct amd_iommu *iommu, u64 f)
            79	{
            80		if (!(iommu->cap & (1 << IOMMU_CAP_EFR)))
            81			return false;
            82	
            83		return !!(iommu->features & f);
            84	}
        […]

So if the former is correct, for some reason `FEATURE_GT` or
`FEATURE_PPR` are not enable when running with coreboot.

        $ nl -ba drivers/iommu/amd_iommu_types.h
        […]
            83	/* Extended Feature Bits */
            84	#define FEATURE_PREFETCH	(1ULL<<0)
            85	#define FEATURE_PPR		(1ULL<<1)
            86	#define FEATURE_X2APIC		(1ULL<<2)
            87	#define FEATURE_NX		(1ULL<<3)
            88	#define FEATURE_GT		(1ULL<<4)
            89	#define FEATURE_IA		(1ULL<<6)
            90	#define FEATURE_GA		(1ULL<<7)
            91	#define FEATURE_HE		(1ULL<<8)
            92	#define FEATURE_PC		(1ULL<<9)
            93	
            94	#define FEATURE_PASID_SHIFT	32
            95	#define FEATURE_PASID_MASK	(0x1fULL << FEATURE_PASID_SHIFT)
            96	
            97	#define FEATURE_GLXVAL_SHIFT	14
            98	#define FEATURE_GLXVAL_MASK	(0x03ULL << FEATURE_GLXVAL_SHIFT)
            99	
           100	#define PASID_MASK		0x000fffff
        […]

Probably adding some debug output to find out what features are enabled
and disabled would be good. But maybe somebody already knows the
solution from the above information.


Thanks,

Paul
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://www.coreboot.org/pipermail/coreboot/attachments/20130629/055abbaf/attachment.sig>


More information about the coreboot mailing list