----- Date:

Home LAN 3: RasPi Fileserver

Previously: Router, Beginning

Myra was designed to run my games, including ludicrous stuff involving VM's and 6TB on 8 spindles with areas for having max bandwidth and others low latency...

"It's complicated, k? Don't touch the rabbits foot on disk 3.*

Suffice it to say that i got trapped into serving files off my desktop PC because that was where the space was.

snafu pic

This is snafu, the new RasPi based File/Web/etc server. 20TB of storage for all our digital video, centralized file shares for the LAN, space for backups, and room to run light web services such as staging versions of this website and archive and browsing tools for our 2TB of photographs.

Files #

Designing a new fileserver with the intent that it do just that one job is a novel idea, but I can see some upsides. What we actually need done isn't terribly complicated, once set up I can move our existing things into 3 categories, work/, bulk/, and slow/.

work/ is an SSD on this lovely little USB-SATA cable, 500GB for our text and spreadsheets and stuff.

bulk/ is for the TB's of pictures and my data horde and so on, currently an 8TB single spindle WD drive. Can't beat rust for price. Keeping most of this free for everything thats coming in the next couple years.

slow/ is this 8TB PSMR drive i got last year for backups which is astonishingly slow to write. I suspect reformatting it and tuning the filesystem parameters could help a lot but for a nearline backup disk there's no reason to change it out and its on hand.

I also have an older 4TB disk on for the bulk of the Video library that moved from myra; I'd wanted to keep it as an offline/swap backup volume but the bandwidth of copying videos to it vs across the network, plus its age, makes it a better choice to go buy a new offline backup disk now.

That disk is now vids/ and the Media Archive is this split between two volumes, which was a problem for the current configuration of the TVs, but corrected with the new media library HTTP server.

Media #

The media archive had been served via SMB/CIFS whatever the $#%^& its called now, "Windows 7 sharing". Because Myra had had earlier clients even more fucked up than cracktop serve at some points in its life, id done things to it with registry hacks and whatever was necessary... Encryption off etc; i don't even recall and didn't want to go back and fix it once it stopped working for new clients.

The better solution is to better serve the media archive, in this case via HTTP off the file server, where i can build a virtual directory tree with symlinks among however many volumes i need to. I know this can be done on windows but its nicer with Unix anyway :)

Now that snafu is there with an always-up webserver and so on, it will be natural for it to also serve the stage site for this blog, and a nice browsing interface for the photo archive, and so on. That's how these things go.

When "should i throw another server at this job or wedge it into and existing one" decisions meant $1k in new hardware, it was harder to do. Today, a RasPi 3 is pretty capable (if low bandwidth) and less than $100 with all the ancillary necessities ... that changes things.

I'm also already looking into MQTT on this system because I have a few sensors and I want more, and to do more with their data... seems a natural place and way to do that.

Setup #

Building out the server is pretty easy. Starting out from a vanilla Raspibian image:

Plug all the drives in and do the basic setup (hostname, install emacs, the critical stuff). Format the volumes as you like, and then set them up in /etc/fstab so they mount automatic at boot.

i@snafu:~ $ cat /etc/fstab
proc            /proc           proc    defaults          0       0
PARTUUID=5e3da3da-01  /boot           vfat    defaults          0       2
PARTUUID=5e3da3da-02  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
UUID=ff1d2629-0124-4a8e-91b2-0adc8f899f37    /work      ext4          defaults,noatime  0 0
UUID=2207cd2f-3bc1-4b39-8109-4945e36d06f3    /bulk      ext4          defaults,noatime  0 0
UUID=04D20AA6D20A9BD4    /slow      ntfs          rw,uid=1000,gid=1000,noatime  0 0
UUID=8EB83CE9B83CD187    /vids      ntfs          rw,uid=1000,gid=1000,noatime  0 0

lsblk -f will give you UUID numbers for the filesystems on your disks.

Samba #

Samba setup is almost as trivial, the important parts here are setting samba to use user "pi" for "guest" shares, and the setup of the shares themselves.

in the Authentication section of /etc/samba/smb.conf, add:

   security = user
   guest account = pi

just after the existing line

   map to guest = bad user

Then at the end of the file, add sections like these to define the shares:

    path = /work
    read only = no
    browseable = yes
    guest ok = yes

    path = /bulk
    read only = no
    browseable = yes
    guest ok = yes

    path = /slow
    read only = yes
    browseable = yes
    guest ok = yes

    path = /vids
    read only = no
    browseable = yes
    guest ok = yes

restart samba sudo /etc/init.d/smbd restart and the shares should be available to clients, without passwords.

All pretty self explanatory and the Samba documentation is very good.

Apache #

sudo apt-get install apache2 and then add the following to /etc/apache2.conf, after the other <Directory> sections:

<Directory /home/pi/snafu.lan/>
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted

restart apache sudo /etc/init.d/apache2 restart and the web server should have a new root... create the directory, populate it, etc.

I've got a basic index page with a couple of links up, symlinks to the file shares so those files are available over http, and the media archive is handled with this directory:

pi@snafu:~ $ ls -la snafu.lan/v/

lrwxrwxrwx 1 pi pi   18 Jan  8 12:02 0new-Movies -> /vids/0new-Movies/
lrwxrwxrwx 1 pi pi   17 Jan  8 12:02 DaddyStuff -> /vids/DaddyStuff/
lrwxrwxrwx 1 pi pi   20 Jan  8 12:03 Documentaries -> /vids/Documentaries/
lrwxrwxrwx 1 pi pi   20 Jan  8 12:03 Movies -> /bulk/Videos/Movies/
lrwxrwxrwx 1 pi pi   27 Jan  8 15:42 Movies-Series -> /bulk/Videos/Movies-Series/
lrwxrwxrwx 1 pi pi   25 Jan  8 18:49 Movies-Toon -> /bulk/Videos/Movies-Toon/
lrwxrwxrwx 1 pi pi   20 Jan  8 12:03 StandupComedy -> /vids/StandupComedy/
lrwxrwxrwx 1 pi pi   15 Jan  8 12:03 TV-daddy -> /vids/TV-daddy/
lrwxrwxrwx 1 pi pi   14 Jan  8 12:03 TVshows -> /vids/TVshows/
lrwxrwxrwx 1 pi pi   13 Jan  8 12:03 TVtoon -> /vids/TVtoon/

Which makes everything available to the TV Pis as http://snafu.lan/v/[whatever] and hides from them the fact that they may be coming off different disks.

Next: TV Pis