Myra was designed to run my games, including ludicrous stuff involving VM's and 6TB on 8 spindles with areas for having max bandwidth and others low latency...
"It's complicated, k? Don't touch the rabbits foot on disk 3.*
Suffice it to say that i got trapped into serving files off my desktop PC because that was where the space was.
This is snafu, the new RasPi based File/Web/etc server. 20TB of storage for all our digital video, centralized file shares for the LAN, space for backups, and room to run light web services such as staging versions of this website and archive and browsing tools for our 2TB of photographs.
Designing a new fileserver with the intent that it do just that one
job is a novel idea, but I can see some upsides. What we actually need
done isn't terribly complicated, once set up I can move our existing
things into 3 categories,
work/ is an SSD on this lovely little USB-SATA cable, 500GB for our
text and spreadsheets and stuff.
bulk/ is for the TB's of pictures and my data horde and so on,
currently an 8TB single spindle WD drive. Can't beat rust for
price. Keeping most of this free for everything thats coming in the
next couple years.
slow/ is this 8TB PSMR drive i got last year for backups which is
astonishingly slow to write. I suspect reformatting it and
tuning the filesystem parameters could help a lot but for a nearline
backup disk there's no reason to change it out and its on hand.
I also have an older 4TB disk on for the bulk of the Video library that moved from myra; I'd wanted to keep it as an offline/swap backup volume but the bandwidth of copying videos to it vs across the network, plus its age, makes it a better choice to go buy a new offline backup disk now.
That disk is now
vids/ and the Media Archive is this split between
two volumes, which was a problem for the current configuration of the
TVs, but corrected with the new media library HTTP server.
The media archive had been served via SMB/CIFS whatever the $#%^& its called now, "Windows 7 sharing". Because Myra had had earlier clients even more fucked up than cracktop serve at some points in its life, id done things to it with registry hacks and whatever was necessary... Encryption off etc; i don't even recall and didn't want to go back and fix it once it stopped working for new clients.
The better solution is to better serve the media archive, in this case via HTTP off the file server, where i can build a virtual directory tree with symlinks among however many volumes i need to. I know this can be done on windows but its nicer with Unix anyway :)
Now that snafu is there with an always-up webserver and so on, it will be natural for it to also serve the stage site for this blog, and a nice browsing interface for the photo archive, and so on. That's how these things go.
When "should i throw another server at this job or wedge it into and existing one" decisions meant $1k in new hardware, it was harder to do. Today, a RasPi 3 is pretty capable (if low bandwidth) and less than $100 with all the ancillary necessities ... that changes things.
I'm also already looking into MQTT on this system because I have a few sensors and I want more, and to do more with their data... seems a natural place and way to do that.
Building out the server is pretty easy. Starting out from a vanilla Raspibian image:
Plug all the drives in and do the basic setup (hostname, install
emacs, the critical stuff). Format the volumes as you like, and then set
them up in
/etc/fstab so they mount automatic at boot.
i@snafu:~ $ cat /etc/fstab proc /proc proc defaults 0 0 PARTUUID=5e3da3da-01 /boot vfat defaults 0 2 PARTUUID=5e3da3da-02 / ext4 defaults,noatime 0 1 # a swapfile is not a swap partition, no line here # use dphys-swapfile swap[on|off] for that UUID=ff1d2629-0124-4a8e-91b2-0adc8f899f37 /work ext4 defaults,noatime 0 0 UUID=2207cd2f-3bc1-4b39-8109-4945e36d06f3 /bulk ext4 defaults,noatime 0 0 UUID=04D20AA6D20A9BD4 /slow ntfs rw,uid=1000,gid=1000,noatime 0 0 UUID=8EB83CE9B83CD187 /vids ntfs rw,uid=1000,gid=1000,noatime 0 0
lsblk -f will give you UUID numbers for the filesystems on your
Samba setup is almost as trivial, the important parts here are setting samba to use user "pi" for "guest" shares, and the setup of the shares themselves.
Authentication section of
security = user guest account = pi
just after the existing line
map to guest = bad user
Then at the end of the file, add sections like these to define the shares:
[work] path = /work read only = no browseable = yes guest ok = yes [bulk] path = /bulk read only = no browseable = yes guest ok = yes [slow] path = /slow read only = yes browseable = yes guest ok = yes [vids] path = /vids read only = no browseable = yes guest ok = yes
sudo /etc/init.d/smbd restart and the shares should be
available to clients, without passwords.
All pretty self explanatory and the Samba documentation is very good.
sudo apt-get install apache2 and then add the following to
/etc/apache2.conf, after the other
<Directory /home/pi/snafu.lan/> Options Indexes FollowSymLinks AllowOverride None Require all granted </Directory>
sudo /etc/init.d/apache2 restart and the web server
should have a new root... create the directory, populate it, etc.
I've got a basic index page with a couple of links up, symlinks to the file shares so those files are available over http, and the media archive is handled with this directory:
pi@snafu:~ $ ls -la snafu.lan/v/ lrwxrwxrwx 1 pi pi 18 Jan 8 12:02 0new-Movies -> /vids/0new-Movies/ lrwxrwxrwx 1 pi pi 17 Jan 8 12:02 DaddyStuff -> /vids/DaddyStuff/ lrwxrwxrwx 1 pi pi 20 Jan 8 12:03 Documentaries -> /vids/Documentaries/ lrwxrwxrwx 1 pi pi 20 Jan 8 12:03 Movies -> /bulk/Videos/Movies/ lrwxrwxrwx 1 pi pi 27 Jan 8 15:42 Movies-Series -> /bulk/Videos/Movies-Series/ lrwxrwxrwx 1 pi pi 25 Jan 8 18:49 Movies-Toon -> /bulk/Videos/Movies-Toon/ lrwxrwxrwx 1 pi pi 20 Jan 8 12:03 StandupComedy -> /vids/StandupComedy/ lrwxrwxrwx 1 pi pi 15 Jan 8 12:03 TV-daddy -> /vids/TV-daddy/ lrwxrwxrwx 1 pi pi 14 Jan 8 12:03 TVshows -> /vids/TVshows/ lrwxrwxrwx 1 pi pi 13 Jan 8 12:03 TVtoon -> /vids/TVtoon/
Which makes everything available to the TV Pis as
http://snafu.lan/v/[whatever] and hides from them the fact that they
may be coming off different disks.
Next: TV Pis