Synology 1812 – Going from 8x2TB to 8x3TB

So recently I decided to give the Synology 1812+ unit a try, origionally it was configured with 8x2TB HDD’s, but at this point I have run out of space, so its time to upgrade.

Since I went with Raid-5 instead of Synology’s SHR raid solution, my RAID Volume will not grow in size until all 8 drives have been replaced (Raid 5 uses all free space of the smallest drives multiplied by the number of drives in the system, less the parity drives).

So here we go…

3.21.13
1) Removed first drive from array. Everyting still running normal, except the storage manager shows ‘degraded,’ and the Synology unit itself is making a really loud, annoying, beep.

2) Put in the new drive, then went to the “Manage” section of the Synology Storage Manager. Then to repair…selected the new disk, then apply.

It has begain checking parity consistency and rebuilding the array. Will update here as things progress. (Approx 9PM).

Transmission OSX + many many torrents (lets test it to the limit)

So lately I’ve been trying to find a torrent client that can handle thousands of torrents at a time and that is capable of downloading data in place of the .torrent file. For this I chose transmission for os x.

I have my music setup so that it’s

/media/Music/Aritst/<type>/Albums

So now I’m able to use the collector at various torrent sites to download entire artists .torrent files at once. Once they are in the proper place where they will permenantly reside, then, using the a search, I locate all .torrent files in my Music directory and drag/drop them over into Transmission.

Transmission then loads all the .torrent files (removing the .torrent at the same time), and begins downloading the data to the directory in which it found the .torrent.

Currently the setup is a Mac Mini (late 2009) w/8GB RAM and gigabit uplink to my network.

It’s been laggy as hell, I can’t even get it to respond long enough to tweak the settings.

Just for a little background the network is setup as such:

Internet Uplink -> pfSense router -> gigabit switch -> Servers/Computers

The data is all being redundantly stored on a Synology 1812+ NAS disk station in RAID-5 format.

Update 1: So eventually I found out that the weakest link currently was my internal HDD in the Mac Mini that I’m using for torrenting. Whenever there is heavy throughput to the internal HDD, the mini will freeze up. So instead of having the files cache in the ~/Downloads directory, I’m not going to have them cache, but instead just write directly to the fileserver.

Update 2: Writing directly to the fileserver doesn’t work because it’s pre-allocating all the files on the fileserver, and thus having to transfer the data for each downloaded torrent twice to the fileserver, slowing everyting way down. Instead I’ll try having the incoming data cache on an external USB 3.0 disk.

Update 3: Everything working much better now that files are being cached on the external USB disk. A .torrent is loaded, the files are pre-allocated locally on the USB drive, the transfer is completed, then at high speed (40-50MB/sec) the data is uploaded to the fileserver and removed from the USB HDD. When a torrent finishes and the data is being uploaded to the fileserver, the incoming transfer rate drops to nearly 0 (but this isn’t a big deal as it takes less than a minute for the data to be copied over and the downloads to resume). 

Update 4: Disabled the Web GUI to see if it makes things any quicker, seems to have amde things slightly faster.

Update 5: Set it up so that when a torrent finishes downloading, it completes after 1 minute instead of seeding. First lets get everything downloaded, then we’ll worry about seeding. Currently at 1500 torrents loaded simultaneously into transmission. Whenever a torrent finishes, Transmission lags up really bad while the data is being copied over to the fileserver. You cannot click anything or use the interface, and download stop downloading. While downloading you can navigate the interface as usual, though it is quite slow and choppy at times. For example, when switching between All, Active, Downloading, etc download lists, sometimes the full list won’t load (ill switch from Active to All, but the list still only displays my currently active transfers). Also wish there was a way to filter out only torrents that are still in progress (i.e. I don’t want to see all the torrents that are complete).

Update 6: Increased max simultaneous torrents to 40 and decreased the time it takes before a download is considered stalled to 1 minute. Peer is set to 80. Getting 12-13MB/sec download bursts, over 1TB transfer yesterday. Currently at ~3000 torrents loaded simultaneously into transmission.

Update 7: Increased max peers to 120 as there are many torrents running with only one peer and my downstream is no longer being saturated. It appears that Transmission will allow more than your max number of running downloads if it pauses a download because its stalled, then resumes it because it became available again. So even though I have my max concurrent downloads set to 40, there’s 80 torrents running right now. Currently at ~5000 torrents loaded simultaneously to transmission. Still saturating my downlink (though sporadically). Not going to load any more torrents until some of the currently loaded ones get fnished up. Switching views on the interface has become extremely slow, laggy, and seldom works corretly. I now have to restart Transmission every now and then if I want to add files or really check the status.

Update 8: Another day, nearly another TB transferred. Now running around 8k torrents, transmission still slow as hell.

status_rrd_graph_img

 

Here’s traffic for the last week. Note that these are averaged, thus you don’t see all the 12-13MB/sec bursts. When watching a live monitor, I see that it normally goes from 0 to 12MB then back to 0.

Synology 1812

So after a few days of deliberation, I finally decided to buy a Synology DS1812+

I received the unit a few days ago from amazon. I immediately loaded it up with 4x2TB and 4×1.5TB drives and configured it in a RAID-5 array (this way when I need more space I can swap the 4×1.5TB for 4x2TB drives).

It took the unit days to complete the parity check, but other than that setup has been a total breeze. I'm currently sending stuff over from a MacMini server I have at 50-80MB/sec (over single GigE).

ipkg

After a little research I found the easiiest way to add functionality to the underlying BusyBox linux operating system that the Synology 1812 runs is to add ipkg.

Once added I proceeded to install lftp, nano, as well as a couple other tools.

<pre> This is a test </pre>