M.2 NVMe SSDs in the 2023 Mac Pro. Use a Sonnet M.2 4x4 PCIe card

This is an ongoing issue seen by many with any nvme storage in pcie slots on Mac Pros 2019 and 2023 (7,1 and 8,1). It happens on direct passthrough cards sometimes but nearly always if there is a apple raid set built or a pcie card using a controller to support multiple nvme drives is used. On my 2019, I can restore the volume by shutting down, performing an SMC reset, booting into recovery, running First Aid in disk utility on the nvme raid set, and then rebooting. This works but sometimes I have to do it twice.

There have been multiple posts about this issue but Apple seems to be ignoring it.

The configuration utility adds up the number of lanes you have assigned to A and to B and tells you the percentage. Stay under 100 percent on each side and you should never face any slowdowns.

One can also modestly over-configure, but I expect you will not want to do that.

Install PCIe cards in your Mac Pro (2023) - Apple Support

scroll down for information on "Expansion Slot Utility" and an exact tally of how many lanes are supplied:

PCIe bandwidth

The M2 Ultra chip provides 32 lanes of PCIe gen 4 to the system, with 8 lanes dedicated to the internal SSD. The M2 Ultra chip connects to the PCIe slots through a PCIe switch and provides 24 lanes of gen 4 bandwidth. Pool A provides a maximum of 16 lanes of gen 4 bandwidth and Pool B provides a maximum of 8 lanes of gen 4 bandwidth.

These built-in components are connected to the system through the PCIe switch and assigned to Pool B:

• SATA controller
• 10Gb Ethernet controllers 
• Wireless (Wi-Fi, Bluetooth) controller
• USB-A port
• PCIe Slot 7 (Apple I/O card)

This means that a system will always show a percentage allocated to Pool B, even if no PCIe cards are installed.

Each built-in Thunderbolt port in Mac Pro is managed by its own controller integrated in the M2 Ultra chip and doesn't share bandwidth with the PCIe slots.

.

they're newest card is the 8x4 Pcie 4.0

from they're website...* Only in a 2023 Mac Pro (M2 Ultra), the initial release of macOS 14 (Sonoma) does not properly access the data in the M.2 8x4 PCIe Card. Users with macOS 13 installed should not update to macOS 14 at this time. We expect Apple to fix this anomaly in an update. (The data will not be lost, just inaccessible.) The card will continue to work fine under macOS 14 in an Intel Mac Pro, or with any Mac using a Thunderbolt expansion system.

i was confused when he said newest....he's talking about the 4x4

When I got my 2023 Mac Pro last year, my intention was to use it with the Sonnet M.2 8x4 Silent Gen4 PCIe Card that had just been put on the market. I ordered one and installed it with eight Crucial P3+ 4Tb NVMe sticks, but couldn't get it to work. Sent it back and got a refund, and then got an OWC Accelsior 8M2, and it works perfectly. I had been worried that the fan on the Accelsior card would be audible, but I can't hear it at all (my Mac Pro is about six feet away from my head, but I can't hear it up close to the machine, either). Very happy with it. I'm a music producer, and I keep all of my sample libraries on that card (I have a LOT of content). I also have three OWC 1M2 PCIe cards with single sticks, and they also work perfectly. If my storage requirements continue to increase, I'll get a 4 stick card, or maybe another 8M2, but that's down the road. I also have the Sonnet hard drive carrier with two 18Tb mechanical drives at the top of the machine, for archival storage. Everything works well.

Previously, with my former 2019 Mac Pro, I had four four SSD drive cases from OWC hooked up via Thunderbolt, and a couple of cases with mechanical drives. I really like having everything inside the Mac Pro now, eliminating that rat's nest of cases and wiring.

i have to give you an answer based upon some speculation as this is not documented by Apple at this time.

My suggestion is that you do some consolidation. At this moment it’s unclear how many PCIe lanes the Pro can accommodate, and putting all of those drives online may cause you some issues. What’s becoming apparent with new systems is that it’s no longer of such great benefit to have libraries spread across drives or to have a lot of smaller drives to spread the load. For my part I’m consolidating several smaller drives that had been externals onto 4TB Thunderbolt drives - because whereas before they were USB-3, and not extraordinarily fast, now a Thunderbolt 3 drive like the Glyph Atom Pro will perform at 2800 MB/sec.

The only issue I can foresee is something I don’t have an answer for: if you install a card like the Sonnet 4x4 in one of the two 16x PCIe slots, it will give each nvme drive 4 PCIe lanes, which means you could get the full speed of your nvme drives, I.e. if they are PCIe 4, around 7000 MB/sec sequential reads in practice. But the issue is that it’s unclear how many lanes are available in the Mac Pro. If you allocate all of those to the card, your interfaces (which I think use a lane each) may or may not have issues. Apple support have been unable to answer this one for me. And it’s unclear how many other cards you would be able to use. I have read of other composers on vi-control (a forum for these and other issues) using the Mac Studio Ultra and external TB4 drives with terrific performance and no issues - with the caveat that you still have a limitation on how many drives you could connect and get full performance from them. One of the users there posted a video showing his Studio Ultra playing back upwards of 910 tracks of Kontakt instruments in Cubase, with what appeared to be no issue streaming voices from his two external drives.

It appears also to be possible (according to a Sonnet tech) that if you connected a 4x4 nvme card to an 8x slot that you would get half the performance, which is still more than 3000 MB/sec and so likely fast enough for sample streaming. So that would mean you could use two of them, possibly. Remember, though, that folks here have been reporting issues using nvme’s on cards when the nvme’s were not formatted on the new systems. And also keep in mind if you were considering the Sonnet 8x4 that it uses a PCIe switch, and it’s unclear how that would react with the needs of sample streaming.

I hope this helps.

I recently installed the Sonnet M.2 8x4 Silent Gen4 PCIe card into my Mac Pro M2. The card is inserted into the x16 PCI slot and contains eight 4TB SABRENT Rocket 4 Plus-G M.2 drives, along with the provided auxiliary power cable. I configured these eight drives into a RAID 0 setup with a 32K block size, resulting in a combined total capacity of 32TB. Apart from the provided Thunderbolt accessories PCIe card, no other components are installed. Curiously, the allocation pool shows A at 100% and B at 88%.

However, I'm facing an issue. Whenever I reboot or shut down and then restart, the Sonnet card's RAID array vanishes as if it's not installed at all. I've had to reboot multiple times before the PCIe card and array are recognized again.

I've tried using both Apple RAID and OWC's SoftRAID, but the issue persists. If I put the computer into a complete sleep state, upon waking it, the Mac Pro indicates that the drive array was improperly ejected. I also own the previous Sonnet M.2 4X4 Silent Gen3 PCIe card, and when I reinstall this older card in the Mac Pro 2023, I encounter no issues. It works flawlessly, just as it did in my previous Mac Pro 2019.

Not compatible with sonoma yet...PCIe 4.0

rosindabow, did you ever get the new Sonnet 8x4 card? I have a very similar situation to yours. Recently got a 2023 Mac Pro, using it in the studio with an Apogee Symphony MkII.

Anyway, I did go ahead and buy the new Sonnet 8x4 NVMe card, and put 8 Crucial P3+ 4Tb NVMe sticks on it. Installed it in Slot 2 of the new Mac Pro, but the computer just does not see it. Sonnet included a power cable, which I used. It's an 8 pin connector on the computer side and a 6 pin connector on the PCIe card side (with an extra 2 pin plug not used). I'm assuming this is the right cable to use. The original instructions say to use a 6 pin to 6 pin cable (but that wasn't included). Revised instruction mention and 8 pin cable.

The LED power lights on the Sonnet board are on, but it's not visible in system report. I tried reseating the card, and also tried slot 1, with no luck. I've contacted Sonnet, and they asked me for a .spx report, and I sent that. But, no answers so far. They did send a .pdf for an RMA form if I want to return it.

I had gotten an email from them about the card not being compatible with Sonoma, but I'm running Ventura. I did not receive any subsequent email saying it was okay to go to Sonoma.

If it worked, this would be an impressive card. From my experience, though, I'm thinking it's not ready for prime time. I'd love to hear someone else's report of success.

If I decided to go with another PCIe NVMe option, I'm not sure what I want. The only other 8 NVMe Gen 4 card out there is the OWC product, but it has a fan. I don't want any additional fans in my studio, if I don't have to have them. The other option is to just get two OWC 4M2 four NVMe cards. Sonnet also has a 4 NVMe card, but I don't know if I want to try my luck. I've alway had good luck with OWC. I'm also looking at the fact that the OWC 4x4 cards can work in either a x8 or x16 slot. So if I buy those now, if I ever upgrade to a Gen 4 card (when the technology stabilizes), I can move the OWC 4M2 cards to x8 slots, and use them for additional utility storage.

Need to make a decision, though. I've got these 8 4Tb sticks, and they're the new plan. I really want to reorganize all of my sample libraries as I configure this new machine (upgrading from a 2019 MP).

RAID uses a device driver stored on the drive itself, to be certain the drive is not accessed as if it were a single drive and mucked up. So when starting up, the drives are not available unless/until that device-driver is loaded off the drive and activated.

That is generally a little slower to get started, but if there are ongoing issues, this could be a new bug. You should try to work closely with your RAID software vendors. OWC/SoftRAID, for example, just fixed a bug where certain brand new, factory-formatted drives would crash the system when first accessed. They issued updated Drivers, but some Users had to send their new drives back for re-formatting at the factory.

The Mac Pro can get very close to full performance, full duplex, for every PCIe device you can install. It is common to modestly over-configure the PCIe slots, knowing that EVERY device running full duplex at exactly the same instant is extremely unlikely, and if that does occur, it will only be a momentary slowdown. So the configurator allows to to modestly over-configure.

Grant Bennet-Alder wrote:

<<. At this moment it’s unclear how many PCIe lanes the Pro can accommodate, >>

The Mac Pro can get very close to full performance, full duplex, for every PCIe device you can install. It is common to modestly over-configure the PCIe slots, knowing that EVERY device running full duplex at exactly the same instant is extremely unlikely, and if that does occur, it will only be a momentary slowdown. So the configurator allows to to modestly over-configure.

Yeah, that's kind of like getting a reservation on an airline, paying for your ticket, and then getting bumped because they oversold since it never happens that everyone books this flight. I'm not a fan of hope or "it shouldn't be a problem - like that's really super rare or something" when it comes to an expensive computer that I rely on. But these aren't strictly speaking workstation-class in the way that the previous models were, and that's fine, as long as you know what you are buying. I have a Mac Studio Ultra, and I'm comfortable with what I ask it to do, as it's all within spec with no guesswork, but I felt like if I bought the Pro that I'd ask it to do things that ultimately it wouldn't do at some point because of the number of lanes present.

For Rotating Magnetic drives, speed above 150 M Bytes/sec are exceptional, and the limiting factor is the Drive. (Drive RPM limits how fast data bits can spin under the read-write head.)

USB 3.2 itself is limited to 20 G bits, 20,000 M bits/sec or around 2,000 M Bytes/sec, so the enclosure is not likely to be the limiting factor. Some of those older SSD drives with speeds under 500 G Bytes/sec speeds might just be older, slower SSD drives, speed-limited by the SSD itself. (Some might actually be SATA SSD enclosures holding limited-speed SATA SSD drives.)

A newer SSD device in that same USB 3.2 enclosure might be both larger and faster. (Provided they are not actually SATA enclosures.)

A genuine ThunderBolt enclosure could provide drive speeds of up to 2500 M Bytes/sec.

Yes, NVMe drives and the bus they sit on is far faster. The OWC 1M2 card for US$25 can support one NVMe device, and in your new M2 Mac, with PCIe 4 slots, that x4 can attain speeds of as high as 7,000 M Bytes/sec with an appropriately fast device..

But you repeatedly express the need for space, not speed.

Rosindabow, it’s difficult to get clear answers when people don’t think you need what you are asking for. You will get unbelievable speed from a 4x4 in a 16x PCIe slot - absolutely plenty for your needs. (Source: I’m a composer.) What other devices do you connect to your system?