About MMU mapping on ARM64

[carlocaione]

Facts

On ARM64 we can MMU-map a memory region in two different ways:

• Directly interfacing with the MMU code
• Going through the Zephyr MMU / device MMIO APIs.

Direct interface with MMU code for direct mapping

This is done by the ARM64 MMU code to setup the basic Zephyr regions (text, data, etc..) in:

zephyr/arch/arm64/core/mmu.c

Lines 649 to 679 in bfec3b2

	static const struct arm_mmu_flat_range mmu_zephyr_ranges[] = {
	/* Mark the zephyr execution regions (data, bss, noinit, etc.)
	* cacheable, read-write
	* Note: read-write region is marked execute-never internally
	*/
	{ .name = "zephyr_data",
	.start = _image_ram_start,
	.end = _image_ram_end,
	.attrs = MT_NORMAL | MT_P_RW_U_NA | MT_DEFAULT_SECURE_STATE },
	/* Mark text segment cacheable,read only and executable */
	{ .name = "zephyr_code",
	.start = __text_region_start,
	.end = __text_region_end,
	.attrs = MT_NORMAL | MT_P_RX_U_RX | MT_DEFAULT_SECURE_STATE },
	/* Mark rodata segment cacheable, read only and execute-never */
	{ .name = "zephyr_rodata",
	.start = __rodata_region_start,
	.end = __rodata_region_end,
	.attrs = MT_NORMAL | MT_P_RO_U_RO | MT_DEFAULT_SECURE_STATE },
	#ifdef CONFIG_NOCACHE_MEMORY
	/* Mark nocache segment noncachable, read-write and execute-never */
	{ .name = "nocache_data",
	.start = _nocache_ram_start,
	.end = _nocache_ram_end,
	.attrs = MT_NORMAL_NC | MT_P_RW_U_RW | MT_DEFAULT_SECURE_STATE },
	#endif
	};

but it also used by the soc-specific code to map regions for peripherals that do not support the device MMIO APIs, for example in:

zephyr/soc/arm64/qemu_cortex_a53/mmu_regions.c

Lines 11 to 22 in bfec3b2

	static const struct arm_mmu_region mmu_regions[] = {
	MMU_REGION_FLAT_ENTRY("GIC",
	DT_REG_ADDR_BY_IDX(DT_INST(0, arm_gic), 0),
	DT_REG_SIZE_BY_IDX(DT_INST(0, arm_gic), 0),
	MT_DEVICE_nGnRnE | MT_P_RW_U_NA | MT_DEFAULT_SECURE_STATE),
	MMU_REGION_FLAT_ENTRY("GIC",
	DT_REG_ADDR_BY_IDX(DT_INST(0, arm_gic), 1),
	DT_REG_SIZE_BY_IDX(DT_INST(0, arm_gic), 1),
	MT_DEVICE_nGnRnE | MT_P_RW_U_NA | MT_DEFAULT_SECURE_STATE),
	};

This mapping is done directly in the MMU driver code and it is usually a direct (1:1) mapping.

Using the device MMIO (or MMU) APIs

There has been lately a certain effort to make the drivers using the device MMIO APIs. These API are leveraging the Zephyr MMU code to map the physical MMIO region of a peripheral to a virtual memory region automatically at init time (see include/zephyr/sys/device_mmio.h)

In general the mapping is not a direct mapping, but instead the virtual region is carved out from a memory pool of virtual addresses configured using CONFIG_KERNEL_VM_BASE and CONFIG_KERNEL_VM_SIZE.

Problems

There are several.

1. The two methods are orthogonal, the only point of contact is the MMU driver that is actually doing the mapping.
2. The Zephyr MMU code is using a simple mechanism to keep tracking of the allocated pages that is being bypassed by the direct interface with the MMU code, so in theory could be conflicts.
3. Especially on ARM64 we have (theoretically) plenty of virtual memory so we really would like to do direct mapping for MMIO driver regions but this is not currently possible with the Zephyr MMU code.

Solution?

The easiest one is to give up the direct interface and instead relying exclusively on the Zephyr MMU code. This would force us to give up the 1:1 mapping or adding support for that.

Tagging the main actors involved @dcpleung @npitre @povergoing

[povergoing]

My concern would be, that the MPU should support the MMIO region too, but these MMIO APIs cannot be reused by MPU if it is not a 1:1 mapping design? You can simply consider the MPU as an MMU that only supports 1:1 mapping.

Does the 1:1 mapping or direct mapping mean virt_addr = phy_addr, IIUC? I didn't learn why the APIs in device_mmio.h needs non-direct mapping since the Zephyr is designed to be a single-memory-space OS.

I am not sure if it is suitable or how difficult it is. Is it possible that we re-use the kernel partitions? Add mmu_zephyr_ranges and all (or part of them marked by some label) peripherals regions defined in DTS into kernel partitions so that MMU or MPU could only consider how to fulfill the kernel partitions. Also, the APIs in device_mmio.h do nothing but add the region into kernel partitions.

[carlocaione]

My concern would be, that the MPU should support the MMIO region too, but these MMIO APIs cannot be reused by MPU if it is not a 1:1 mapping design? You can simply consider the MPU as an MMU that only supports 1:1 mapping.

Well, I was not aware of that and this is definitely concerning (:hankey:)

Does the 1:1 mapping or direct mapping mean virt_addr = phy_addr, IIUC?

Yes.

I didn't learn why the APIs in device_mmio.h needs non-direct mapping since the Zephyr is designed to be a single-memory-space OS.

I think @dcpleung could shed some light on this.

But the point is that when a physical address needs to be mapped using z_phys_map() the destination virtual address is obtained by a pool of virtual address and then mapped using arch_mem_map(). See:

zephyr/kernel/mmu.c

Lines 736 to 751 in d130160

	/* Obtain an appropriately sized chunk of virtual memory */
	dest_addr = virt_region_alloc(aligned_size, align_boundary);
	if (!dest_addr) {
	goto fail;
	}
	/* If this fails there's something amiss with virt_region_get */
	__ASSERT((uintptr_t)dest_addr <
	((uintptr_t)dest_addr + (size - 1)),
	"wraparound for virtual address %p (size %zu)",
	dest_addr, size);
	LOG_DBG("arch_mem_map(%p, 0x%lx, %zu, %x) offset %lu", dest_addr,
	aligned_phys, aligned_size, flags, addr_offset);
	arch_mem_map(dest_addr, aligned_phys, aligned_size, flags);

so it definitely is not a 1:1 mapping (or AFAICT).

I am not sure if it is suitable or how difficult it is. Is it possible that we re-use the kernel partitions? Add mmu_zephyr_ranges and all (or part of them marked by some label) peripherals regions defined in DTS into kernel partitions so that MMU or MPU could only consider how to fulfill the kernel partitions. Also, the APIs in device_mmio.h do nothing but add the region into kernel partitions.

Uhm, this seems more complicated than adding support for 1:1 in the current API.

[carlocaione]

Well, I was not aware of that and this is definitely concerning (hankey)

Oh well, maybe not. I just checked and when MMU is not present (i.e. you have MPU), the device MMIO APIs are not mapping anything and you are basically back to accessing straight the phys.

[dcpleung]

The device MMIO was introduced before I took over userspace, so the design decision is a bit fuzzy. But IIRC, it is working similar to the Linux Kernel where the MMIO range is not 1:1 mapping in general (at least on x86).

Just wondering what would be the use case for having 1:1 mapping? I can see that it would make debugging easier, but in production, does it matter where the hardware registers are mapped?

[carlocaione]

Just wondering what would be the use case for having 1:1 mapping? I can see that it would make debugging easier, but in production, does it matter where the hardware registers are mapped?

Well, the big issue with Zephyr is that 95% of the drivers are not using the device MMIO API and that means that they are basically accessing the physical address all the times (usually the physical address is retrieved from the DT with the usual DT_REG_ADDR, saved into the config struct and used to access the various registers).

So either you fix the driver adding support for the MMIO API (so the driver uses the virt address instead of phys) or you add a 1:1 mapping leaving the driver unfixed. See for example what happened here #46443 (comment).

This is a huge problem IMHO.

[dcpleung]

Well, the big issue with Zephyr is that 95% of the drivers are not using the device MMIO API and that means that they are basically accessing the physical address all the times (usually the physical address is retrieved from the DT with the usual DT_REG_ADDR, saved into the config struct and used to access the various registers).

So either you fix the driver adding support for the MMIO API (so the driver uses the virt address instead of phys) or you add a 1:1 mapping leaving the driver unfixed. See for example what happened here #46443 (comment).

This is a huge problem IMHO.

Driver not using MMIO API is indeed a huge issue when dealing with MMU as those addresses by default are not accessible. Though I was asking what were the use cases when using the MMIO API. I would assume a proper MMU implementation allows I/O addresses to be mapped into virtual space.

[carlocaione]

Though I was asking what were the use cases when using the MMIO API.

Oh right, I probably explained myself badly.

So, If you are using the MMIO API and the driver supports it there is indeed no problem, we are fine in that case even without a 1:1 mapping.

We still have to deal with the case where the driver is not using the MMIO API. In this case for ARM64 we are bypassing this problem by directly creating the 1:1 mapping using the MMU driver but entirely bypassing the Zephyr MMU code. So my suggestion was for this second case: removing the direct interface with the MMU driver and instead relying on the Zephyr MMU code to create the 1:1 mapping for all the driver still not supporting MMIO API.

[dcpleung]

Maybe we can convert those drivers to use the device MMIO API when they are being included? TBH, anything we do now to make those non-"device MMIO API" enabled devices work would be a stop-gap effort. So I think the proper way going forward is to convert them to use device MMIO API. Though... I don't know how many you will need to do at the moment. Could you hazard a guess on what you need for your development at the moment?

[povergoing]

removing the direct interface with the MMU driver and instead relying on the Zephyr MMU code to create the 1:1 mapping for all the driver still not supporting MMIO API.

Cool, that means, if we want MPU to support MMIO API instead of a big device region, we can extend the non-MMU case?

[carlocaione]

Maybe we can convert those drivers to use the device MMIO API when they are being included? TBH, anything we do now to make those non-"device MMIO API" enabled devices work would be a stop-gap effort. So I think the proper way going forward is to convert them to use device MMIO API.

Yes, this is indeed what I'm trying to do while reviewing new drivers submission: convince people to use MMIO API.

I don't know how many you will need to do at the moment. Could you hazard a guess on what you need for your development at the moment?

I don't need any for my development but: (1) this must be considered for new drivers submission and (2) this is part of a cleanup work to remove the mmu_regions for good.

About the point (2) in general having the two methods (the MMIO API and the direct mapping using mmu_regions) is confusing for developers and prone to errors in the long term (what if the MMIO API is mapping to a virt address that is already mapped by the MMU driver for example?).

[carlocaione]

Cool, that means, if we want MPU to support MMIO API instead of a big device region, we can extend the non-MMU case?

Possibly? But the MPU case is definitely easier (and more limited since you have a limited number of slots) and I'm not sure if going through the MMIO API is worth it