
ijustpostwhenimhi
1565
44
3

All are enterprise 12Gb/s SAS SSDs
2x 800GB SSDs, mirrored as a ZFS special vdev.
The vdev stores metadata and small files. Most of my files are large, so it's mostly for metadata.
1x 1.6TB SSD assigned as L2ARC, i.e. read cache.
Initial testing showed a noticable latency improvement when crawling directories.
Trying my best to work with this footprint and still get all the cables routed neatly >.>
There will be one final revision after this, for external SAS port and power input.

Positioning the 2 front SSDs was tricky with the Pi power cable in the way. I may try a right angle USB-c adapter if it makes cable mgmt easier but for now this works.
I would have liked to place the rear SSD where the air flows through the fins, but I just don't have enough room. The good news is, during testing it didn't take much to keep all 3 drives in the low 30c's with a single silent 80mm fan.

Speaking of fans. I updated their placement to focus on the hot spots. This is the left half, but the right half was also updated.
I'll be using Noctua 40x20mm

The cards aren't fastened here, just a placement test

@StackMySwitchUp
Next major addition will be a power sequencing circuit for the PCIe cards. I'm waiting on the mouser order.
The PCIe lane error counter only goes up during reboots, and never during normal operation. I suspect it may have something to do with the 12v rail not being controlled by the pi so the cards don't power cycle with the pi.

Cat tax
EnhydraLutra
Looks awesome! I always wanted to try a PiNAS for myself, but then I just keep using my previous gaming computer as my NAS/HTPC.
fallswinkingtothefloor
Your what project
ijustpostwhenimhi
Raspberry Pi Network Attached Storage
fallswinkingtothefloor
Good, thought I was mishearing
UnitConversionBot
80mm ≈ 3 1/8 inches
UnitConversionBot
30c ≈ 86 ° Fahrenheit or 300 Kelvin
MrNohbdy
PwnageHobo
I'm no stranger to Pi project names, but I can see you know exactly how I was pronouncing "PiNAS" in my head
FrogBotherer
zer0vector
I mean cool project and if you like it that's all that matters, but grabbing a cheap optiplex and cramming some drives in there ends up with a much more stable end product. I've run my nas off a variety of PCs, miniPCs, raspis, etc over the years and just basic simplicity has won out as the biggest factor.
ijustpostwhenimhi
StackMySwitchUp
I bet you can do something with the Pi GPIO and a simple relay to help with the power sequencing. Love how neat it goes together, you made something remarkable. It has me itching to make some changes to my system, but i need to finish about 10 other projects, and it aint broke, so why should I fix it..
ijustpostwhenimhi
I was going to go with a solid state relay, but even a small 4A DIP6 was $11 on mouser, and I didn't want to go mechanical. Thankfully the RPi FFC connector has a PCIe enable signal, and the official m.2 hat has a test point for it. Gonna go with a high-side PNP mosfet for the switch, and a BJT to invert the PCIe enable signal.
ijustpostwhenimhi
2/2 scratch that, I found some 4.5A DIP SSRs for $5, and I'm running out of case space for a more complex circuit lol https://www.mouser.com/ProductDetail/Coto-Technology/CT128?qs=vkMqrgFpRagofYxb58SuRA%3D%3D Didn't see these before because Mouser was not classifying them as "Through-Hole", so my search filters hid them
StackMySwitchUp
I wouldn't limit myself to through-hole components anyway, you're at a point where a custom circuit board starts making sense, perhaps even a HAT extension
ijustpostwhenimhi
True. That's gonna be for later, I just wanted something I could breadboard and then stick on a protoboard, but I could integrate power sequencing into my planned gen3 switch hat, (also considering a CM5 expansion board). Broadcom approved my NDA, so I've got access to the PEX8712 docs and design references now :D
StackMySwitchUp
Surprising they would agree to that, but nice!
smellsmiketeenspirit
#5 Do you have a blog post about your setup ? I don't understand what the three bottom cards are for.
ijustpostwhenimhi
I've been meaning to put all the post entries in a fav folder, but for now #nas https://imgur.com/t/nas is mostly me.
The cards from top,
Asmedia ASM2812 PCIe gen3 switch, configured as x4:x4:x4,
Broadcom 9400-8i SAS I/O Controller,
Intel X710 10GbE NIC
lddiaz
I did a PiNAS. It would freeze up every 6 months or so. Open Media Vault. Had to re-image it every time. The last time, it failed in the middle of a movie. I put the kibosh in the whole thing. I now have Windows 2019 running on a micro form factor HP. No problems for 3 years.
Hope you have better luck.
smellsmiketeenspirit
Did you find the culprit ? Disk drivers, open media vault raid stack, OS, hardware ?
RowanUnderwood
Meanwhile I'm sitting at the bottom of a 43 disk nightmare. Enjoy your beautiful simplicity!
ColmCorbec
Something about the first partition showing as '203tb' makes my eye twitch and I'm not sure why.
ijustpostwhenimhi
The main bay will be 12 8TB, raidz2 so 80TB usable. That should be good for my needs for quite some time, but the backplane I'm using supports daisy chaining, and I have 2 more, so maybe more in the future :) 43 is a lot though. Do you have cold backups? Still working out a solution for that. Tape writers are hella expensive
RowanUnderwood
I do actually have a cold backup also - not of the entire 150tb though. No offsite either. If she burns she burns.
smellsmiketeenspirit
Did I stumble upon a r/datahoarder conversation ? Don't mind me, just taking notes.
RowanUnderwood
Guilty
BlindMeach
How much?!?!
RowanUnderwood
How much money? - no clue and do not want to know. I never buy a drive that isn't at or under $15/tb though. How much electricity? Don't want to know that either.
BlindMeach
Ooh no- not costing- how much storage?! I keep all my media on less than 8tb, you must have some hefty files! If I ever get to upgrade to uhd, all that will have to change though:)
RowanUnderwood
My current total is somewhere around 550TB - that is 3 copies of important stuff - 2 copies of everything else. Just bought 6 more 20tb drives that will be added as I have time - bringing me up to 670.
RowanUnderwood
A lot of my stuff is down to not trusting internet archive to be around forever, so I'm backing up tons of retro game stuff.