Skip to content

Commit bdf5309

Browse files
committed
Merge branch 'pci/host-vmd' into next
* pci/host-vmd: x86/PCI: VMD: Move VMD driver to drivers/pci/host x86/PCI: VMD: Synchronize with RCU freeing MSI IRQ descs x86/PCI: VMD: Eliminate index member from IRQ list x86/PCI: VMD: Eliminate vmd_vector member from list type x86/PCI: VMD: Convert to use pci_alloc_irq_vectors() API x86/PCI: VMD: Allocate IRQ lists with correct MSI-X count PCI: Use positive flags in pci_alloc_irq_vectors() PCI: Update "pci=resource_alignment" documentation Conflicts: drivers/pci/host/Kconfig drivers/pci/host/Makefile
2 parents 69a06e4 + 181ffd1 commit bdf5309

File tree

9 files changed

+66
-67
lines changed

9 files changed

+66
-67
lines changed

Documentation/PCI/MSI-HOWTO.txt

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -94,14 +94,11 @@ has a requirements for a minimum number of vectors the driver can pass a
9494
min_vecs argument set to this limit, and the PCI core will return -ENOSPC
9595
if it can't meet the minimum number of vectors.
9696

97-
The flags argument should normally be set to 0, but can be used to pass the
98-
PCI_IRQ_NOMSI and PCI_IRQ_NOMSIX flag in case a device claims to support
99-
MSI or MSI-X, but the support is broken, or to pass PCI_IRQ_NOLEGACY in
100-
case the device does not support legacy interrupt lines.
101-
102-
By default this function will spread the interrupts around the available
103-
CPUs, but this feature can be disabled by passing the PCI_IRQ_NOAFFINITY
104-
flag.
97+
The flags argument is used to specify which type of interrupt can be used
98+
by the device and the driver (PCI_IRQ_LEGACY, PCI_IRQ_MSI, PCI_IRQ_MSIX).
99+
A convenient short-hand (PCI_IRQ_ALL_TYPES) is also available to ask for
100+
any possible kind of interrupt. If the PCI_IRQ_AFFINITY flag is set,
101+
pci_alloc_irq_vectors() will spread the interrupts around the available CPUs.
105102

106103
To get the Linux IRQ numbers passed to request_irq() and free_irq() and the
107104
vectors, use the following function:
@@ -131,7 +128,7 @@ larger than the number supported by the device it will automatically be
131128
capped to the supported limit, so there is no need to query the number of
132129
vectors supported beforehand:
133130

134-
nvec = pci_alloc_irq_vectors(pdev, 1, nvec, 0);
131+
nvec = pci_alloc_irq_vectors(pdev, 1, nvec, PCI_IRQ_ALL_TYPES)
135132
if (nvec < 0)
136133
goto out_err;
137134

@@ -140,23 +137,22 @@ interrupts it can request a particular number of interrupts by passing that
140137
number to pci_alloc_irq_vectors() function as both 'min_vecs' and
141138
'max_vecs' parameters:
142139

143-
ret = pci_alloc_irq_vectors(pdev, nvec, nvec, 0);
140+
ret = pci_alloc_irq_vectors(pdev, nvec, nvec, PCI_IRQ_ALL_TYPES);
144141
if (ret < 0)
145142
goto out_err;
146143

147144
The most notorious example of the request type described above is enabling
148145
the single MSI mode for a device. It could be done by passing two 1s as
149146
'min_vecs' and 'max_vecs':
150147

151-
ret = pci_alloc_irq_vectors(pdev, 1, 1, 0);
148+
ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES);
152149
if (ret < 0)
153150
goto out_err;
154151

155152
Some devices might not support using legacy line interrupts, in which case
156-
the PCI_IRQ_NOLEGACY flag can be used to fail the request if the platform
157-
can't provide MSI or MSI-X interrupts:
153+
the driver can specify that only MSI or MSI-X is acceptable:
158154

159-
nvec = pci_alloc_irq_vectors(pdev, 1, nvec, PCI_IRQ_NOLEGACY);
155+
nvec = pci_alloc_irq_vectors(pdev, 1, nvec, PCI_IRQ_MSI | PCI_IRQ_MSIX);
160156
if (nvec < 0)
161157
goto out_err;
162158

Documentation/kernel-parameters.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3032,6 +3032,10 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
30323032
PAGE_SIZE is used as alignment.
30333033
PCI-PCI bridge can be specified, if resource
30343034
windows need to be expanded.
3035+
To specify the alignment for several
3036+
instances of a device, the PCI vendor,
3037+
device, subvendor, and subdevice may be
3038+
specified, e.g., 4096@pci:8086:9c22:103c:198f
30353039
ecrc= Enable/disable PCIe ECRC (transaction layer
30363040
end-to-end CRC checking).
30373041
bios: Use BIOS/firmware settings. This is the

arch/x86/Kconfig

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2744,19 +2744,6 @@ config PMC_ATOM
27442744
def_bool y
27452745
depends on PCI
27462746

2747-
config VMD
2748-
depends on PCI_MSI
2749-
tristate "Volume Management Device Driver"
2750-
default N
2751-
---help---
2752-
Adds support for the Intel Volume Management Device (VMD). VMD is a
2753-
secondary PCI host bridge that allows PCI Express root ports,
2754-
and devices attached to them, to be removed from the default
2755-
PCI domain and placed within the VMD domain. This provides
2756-
more bus resources than are otherwise possible with a
2757-
single domain. If you know your system provides one of these and
2758-
has devices attached to it, say Y; if you are not sure, say N.
2759-
27602747
source "net/Kconfig"
27612748

27622749
source "drivers/Kconfig"

arch/x86/pci/Makefile

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,6 @@ obj-y += bus_numa.o
2323
obj-$(CONFIG_AMD_NB) += amd_bus.o
2424
obj-$(CONFIG_PCI_CNB20LE_QUIRK) += broadcom_bus.o
2525

26-
obj-$(CONFIG_VMD) += vmd.o
27-
2826
ifeq ($(CONFIG_PCI_DEBUG),y)
2927
EXTRA_CFLAGS += -DDEBUG
3028
endif

drivers/pci/host/Kconfig

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -285,4 +285,20 @@ config PCIE_ROCKCHIP
285285
There is 1 internal PCIe port available to support GEN2 with
286286
4 slots.
287287

288+
config VMD
289+
depends on PCI_MSI && X86_64
290+
tristate "Intel Volume Management Device Driver"
291+
default N
292+
---help---
293+
Adds support for the Intel Volume Management Device (VMD). VMD is a
294+
secondary PCI host bridge that allows PCI Express root ports,
295+
and devices attached to them, to be removed from the default
296+
PCI domain and placed within the VMD domain. This provides
297+
more bus resources than are otherwise possible with a
298+
single domain. If you know your system provides one of these and
299+
has devices attached to it, say Y; if you are not sure, say N.
300+
301+
To compile this driver as a module, choose M here: the
302+
module will be called vmd.
303+
288304
endmenu

drivers/pci/host/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,4 @@ obj-$(CONFIG_PCI_HOST_THUNDER_PEM) += pci-thunder-pem.o
3232
obj-$(CONFIG_PCIE_ARMADA_8K) += pcie-armada8k.o
3333
obj-$(CONFIG_PCIE_ARTPEC6) += pcie-artpec6.o
3434
obj-$(CONFIG_PCIE_ROCKCHIP) += pcie-rockchip.o
35+
obj-$(CONFIG_VMD) += vmd.o

arch/x86/pci/vmd.c renamed to drivers/pci/host/vmd.c

Lines changed: 22 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -56,16 +56,11 @@ struct vmd_irq {
5656
/**
5757
* struct vmd_irq_list - list of driver requested IRQs mapping to a VMD vector
5858
* @irq_list: the list of irq's the VMD one demuxes to.
59-
* @vmd_vector: the h/w IRQ assigned to the VMD.
60-
* @index: index into the VMD MSI-X table; used for message routing.
6159
* @count: number of child IRQs assigned to this vector; used to track
6260
* sharing.
6361
*/
6462
struct vmd_irq_list {
6563
struct list_head irq_list;
66-
struct vmd_dev *vmd;
67-
unsigned int vmd_vector;
68-
unsigned int index;
6964
unsigned int count;
7065
};
7166

@@ -76,7 +71,6 @@ struct vmd_dev {
7671
char __iomem *cfgbar;
7772

7873
int msix_count;
79-
struct msix_entry *msix_entries;
8074
struct vmd_irq_list *irqs;
8175

8276
struct pci_sysdata sysdata;
@@ -95,6 +89,12 @@ static inline struct vmd_dev *vmd_from_bus(struct pci_bus *bus)
9589
return container_of(bus->sysdata, struct vmd_dev, sysdata);
9690
}
9791

92+
static inline unsigned int index_from_irqs(struct vmd_dev *vmd,
93+
struct vmd_irq_list *irqs)
94+
{
95+
return irqs - vmd->irqs;
96+
}
97+
9898
/*
9999
* Drivers managing a device in a VMD domain allocate their own IRQs as before,
100100
* but the MSI entry for the hardware it's driving will be programmed with a
@@ -107,9 +107,11 @@ static void vmd_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
107107
{
108108
struct vmd_irq *vmdirq = data->chip_data;
109109
struct vmd_irq_list *irq = vmdirq->irq;
110+
struct vmd_dev *vmd = irq_data_get_irq_handler_data(data);
110111

111112
msg->address_hi = MSI_ADDR_BASE_HI;
112-
msg->address_lo = MSI_ADDR_BASE_LO | MSI_ADDR_DEST_ID(irq->index);
113+
msg->address_lo = MSI_ADDR_BASE_LO |
114+
MSI_ADDR_DEST_ID(index_from_irqs(vmd, irq));
113115
msg->data = 0;
114116
}
115117

@@ -194,16 +196,19 @@ static int vmd_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
194196
struct msi_desc *desc = arg->desc;
195197
struct vmd_dev *vmd = vmd_from_bus(msi_desc_to_pci_dev(desc)->bus);
196198
struct vmd_irq *vmdirq = kzalloc(sizeof(*vmdirq), GFP_KERNEL);
199+
unsigned int index, vector;
197200

198201
if (!vmdirq)
199202
return -ENOMEM;
200203

201204
INIT_LIST_HEAD(&vmdirq->node);
202205
vmdirq->irq = vmd_next_irq(vmd, desc);
203206
vmdirq->virq = virq;
207+
index = index_from_irqs(vmd, vmdirq->irq);
208+
vector = pci_irq_vector(vmd->dev, index);
204209

205-
irq_domain_set_info(domain, virq, vmdirq->irq->vmd_vector, info->chip,
206-
vmdirq, handle_untracked_irq, vmd, NULL);
210+
irq_domain_set_info(domain, virq, vector, info->chip, vmdirq,
211+
handle_untracked_irq, vmd, NULL);
207212
return 0;
208213
}
209214

@@ -213,6 +218,8 @@ static void vmd_msi_free(struct irq_domain *domain,
213218
struct vmd_irq *vmdirq = irq_get_chip_data(virq);
214219
unsigned long flags;
215220

221+
synchronize_rcu();
222+
216223
/* XXX: Potential optimization to rebalance */
217224
raw_spin_lock_irqsave(&list_lock, flags);
218225
vmdirq->irq->count--;
@@ -672,30 +679,19 @@ static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
672679
if (vmd->msix_count < 0)
673680
return -ENODEV;
674681

682+
vmd->msix_count = pci_alloc_irq_vectors(dev, 1, vmd->msix_count,
683+
PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
684+
if (vmd->msix_count < 0)
685+
return vmd->msix_count;
686+
675687
vmd->irqs = devm_kcalloc(&dev->dev, vmd->msix_count, sizeof(*vmd->irqs),
676688
GFP_KERNEL);
677689
if (!vmd->irqs)
678690
return -ENOMEM;
679691

680-
vmd->msix_entries = devm_kcalloc(&dev->dev, vmd->msix_count,
681-
sizeof(*vmd->msix_entries),
682-
GFP_KERNEL);
683-
if (!vmd->msix_entries)
684-
return -ENOMEM;
685-
for (i = 0; i < vmd->msix_count; i++)
686-
vmd->msix_entries[i].entry = i;
687-
688-
vmd->msix_count = pci_enable_msix_range(vmd->dev, vmd->msix_entries, 1,
689-
vmd->msix_count);
690-
if (vmd->msix_count < 0)
691-
return vmd->msix_count;
692-
693692
for (i = 0; i < vmd->msix_count; i++) {
694693
INIT_LIST_HEAD(&vmd->irqs[i].irq_list);
695-
vmd->irqs[i].vmd_vector = vmd->msix_entries[i].vector;
696-
vmd->irqs[i].index = i;
697-
698-
err = devm_request_irq(&dev->dev, vmd->irqs[i].vmd_vector,
694+
err = devm_request_irq(&dev->dev, pci_irq_vector(dev, i),
699695
vmd_irq, 0, "vmd", &vmd->irqs[i]);
700696
if (err)
701697
return err;

drivers/pci/msi.c

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1069,7 +1069,7 @@ static int __pci_enable_msi_range(struct pci_dev *dev, int minvec, int maxvec,
10691069
nvec = maxvec;
10701070

10711071
for (;;) {
1072-
if (!(flags & PCI_IRQ_NOAFFINITY)) {
1072+
if (flags & PCI_IRQ_AFFINITY) {
10731073
dev->irq_affinity = irq_create_affinity_mask(&nvec);
10741074
if (nvec < minvec)
10751075
return -ENOSPC;
@@ -1105,7 +1105,7 @@ static int __pci_enable_msi_range(struct pci_dev *dev, int minvec, int maxvec,
11051105
**/
11061106
int pci_enable_msi_range(struct pci_dev *dev, int minvec, int maxvec)
11071107
{
1108-
return __pci_enable_msi_range(dev, minvec, maxvec, PCI_IRQ_NOAFFINITY);
1108+
return __pci_enable_msi_range(dev, minvec, maxvec, 0);
11091109
}
11101110
EXPORT_SYMBOL(pci_enable_msi_range);
11111111

@@ -1120,7 +1120,7 @@ static int __pci_enable_msix_range(struct pci_dev *dev,
11201120
return -ERANGE;
11211121

11221122
for (;;) {
1123-
if (!(flags & PCI_IRQ_NOAFFINITY)) {
1123+
if (flags & PCI_IRQ_AFFINITY) {
11241124
dev->irq_affinity = irq_create_affinity_mask(&nvec);
11251125
if (nvec < minvec)
11261126
return -ENOSPC;
@@ -1160,8 +1160,7 @@ static int __pci_enable_msix_range(struct pci_dev *dev,
11601160
int pci_enable_msix_range(struct pci_dev *dev, struct msix_entry *entries,
11611161
int minvec, int maxvec)
11621162
{
1163-
return __pci_enable_msix_range(dev, entries, minvec, maxvec,
1164-
PCI_IRQ_NOAFFINITY);
1163+
return __pci_enable_msix_range(dev, entries, minvec, maxvec, 0);
11651164
}
11661165
EXPORT_SYMBOL(pci_enable_msix_range);
11671166

@@ -1187,21 +1186,21 @@ int pci_alloc_irq_vectors(struct pci_dev *dev, unsigned int min_vecs,
11871186
{
11881187
int vecs = -ENOSPC;
11891188

1190-
if (!(flags & PCI_IRQ_NOMSIX)) {
1189+
if (flags & PCI_IRQ_MSIX) {
11911190
vecs = __pci_enable_msix_range(dev, NULL, min_vecs, max_vecs,
11921191
flags);
11931192
if (vecs > 0)
11941193
return vecs;
11951194
}
11961195

1197-
if (!(flags & PCI_IRQ_NOMSI)) {
1196+
if (flags & PCI_IRQ_MSI) {
11981197
vecs = __pci_enable_msi_range(dev, min_vecs, max_vecs, flags);
11991198
if (vecs > 0)
12001199
return vecs;
12011200
}
12021201

12031202
/* use legacy irq if allowed */
1204-
if (!(flags & PCI_IRQ_NOLEGACY) && min_vecs == 1)
1203+
if ((flags & PCI_IRQ_LEGACY) && min_vecs == 1)
12051204
return 1;
12061205
return vecs;
12071206
}

include/linux/pci.h

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1264,10 +1264,12 @@ resource_size_t pcibios_iov_resource_alignment(struct pci_dev *dev, int resno);
12641264
int pci_set_vga_state(struct pci_dev *pdev, bool decode,
12651265
unsigned int command_bits, u32 flags);
12661266

1267-
#define PCI_IRQ_NOLEGACY (1 << 0) /* don't use legacy interrupts */
1268-
#define PCI_IRQ_NOMSI (1 << 1) /* don't use MSI interrupts */
1269-
#define PCI_IRQ_NOMSIX (1 << 2) /* don't use MSI-X interrupts */
1270-
#define PCI_IRQ_NOAFFINITY (1 << 3) /* don't auto-assign affinity */
1267+
#define PCI_IRQ_LEGACY (1 << 0) /* allow legacy interrupts */
1268+
#define PCI_IRQ_MSI (1 << 1) /* allow MSI interrupts */
1269+
#define PCI_IRQ_MSIX (1 << 2) /* allow MSI-X interrupts */
1270+
#define PCI_IRQ_AFFINITY (1 << 3) /* auto-assign affinity */
1271+
#define PCI_IRQ_ALL_TYPES \
1272+
(PCI_IRQ_LEGACY | PCI_IRQ_MSI | PCI_IRQ_MSIX)
12711273

12721274
/* kmem_cache style wrapper around pci_alloc_consistent() */
12731275

0 commit comments

Comments
 (0)