New firmware for my Western Digital “My Book Live” NAS storage device

The WD My Book Live is a NAS device based on Debian Linux, Since Debian stopped supporting this processor (APM82181), the device has received no updates and will probably never, so the next best thing to do in my opinion is to install openWRT.

Before you start

1- Only the first few paragraphs of this tutorial (STEPS 1 THROUGH 6) are the instructions you need, the remaining is just for extra reference and in short you don’t need to read it to have your device running, but I do recommend YOU SKIM THE WHOLE THING BEFORE YOU START.
2- This procedure requires you to take the disk out and install it on a PC to switch the firmware, then put it back
3- The upgrade will delete all your data, You will need to move your data that is already on your WD NAS drive somewhere else while the upgrade is ready.

Step 1: Move any existing data BEFORE TAKING APART.

Move any data you may have on the drive to a temporary location outside the NAS drive. this has to be done before taking the drive apart as the unconventional 64 kB block size of the disk will be nothing but trouble if you want to extract the data while mounting the disk to a linux PC for example.

Step 2: Take the disk apart

I have included photos to help you do that, it is not rocket science.

Step 3: Mount the disk on a linux PC (Windows and MAC should work)

and mount it to a linux PC (Windows might work with software such as etcher, but i have no guarantees).

Step 4: Download the openWRT firmware

Go to the drive’s page on the openwrt website (Here), and download it to your Linux (Or windows) PC

Step 5: Write the firmware to the disk.

Decompress the file, then copy it to the drive with a command similar to the command below, but make 100% sure to replace sdx with your own drive designation

 dd if=/root/wdsata.img of=/dev/sdx bs=64k

Write the firmware to the disk, overwriting it, and effectively loosing any data you did not backup in step 1

Step 6: Put the drive back in the enclosure

Nothing to say here, this is the reverse of step 2

Once it is in the enclosure, you can not just connect it to your router as it in itself has this port defined as 192.168.1.1 and is serving dhcp !

Step 8: Create the data partition

At this stage, your device will boot, but you will need to create/expand the data partition, the partition that should not be overwritten when you upgrade the firmware for example.

opkg update
opkg install gdisk blkid openssh-sftp-server
gdisk -i /dev/sda

As soon as gdisk opens, you may be presented with the following message, if so

Found valid MBR and corrupt GPT. Which do you want to use? (Using the
GPT MAY permit recovery of GPT data.)
 1 - MBR
 2 - GPT
 3 - Create blank GPT

Chose 1 to maintain the 2 partitions we have, Now hit the command (w) to write and confirm, then quit, gdisk has just switched your disk to GPT from MBR, now run gdisk again the same way (gdisk -i /dev/sda)

n for new partition, accept the (3) for partition number, use the number (2097152) for alignment with 4K sector advanced format nearest to the 1GB mark

mkfs.ext4 /dev/sda3
mkdir /share
blkid /dev/sda3
in my case, the UUID was as follows UUID="9643bd00-f117-4074-a252-7ea30a5174e2" yours will certainly be different, so in my fstab i added the following lines near the end
config mount
option target '/share'
option uuid '9643bd00-f117-4074-a252-7ea30a5174e2'
option enabled '1'

Now, network sharing is what i was originally interested in when i got this unit, and it is why I am replacing it’s firmware, so to installing samba

opkg update && opkg install samba4-server luci-app-samba4

Now, add the following line to /etc/passwd to add me as a user to the system

yazeed:*:1000:65534:yazeed:/var:/bin/false

Now, run the command

passwd yazeed
smbpasswd -a yazeed

You are done.

FAQ

Is the hardware and the new openWRT firmware compatible with my 8TB hard drive

Yes it is, I have found many people asking if the hardware supports drives over 2TB, the answer is yes, but you will have to use the GPT rather than the MBR (See steps above)

about the original firmware

What is that vulnerability about

it comes from WDs cloud service, bottom line is that many devices were completely wiped remotely by malicious users and it is unknown if the data itself leaked, so yes, it is very serious

What is the difference between quick factory restore and full factory restore

Quick factory restore is probably what you are looking for, the later seems to do a zero fill on the hard drive after performing a factory restore to disallow data retrieval (For example before you sell it), you can verify this by logging in using SSH, and by the fact that the tool tips state something to that effect.

Inspecting the device

To begin with, I logged in via SSH and inspected some stuff, to enable SSH access on the My Book Live original firmware, you will need to visit a page at a URL such as http://mybooklive/UI/ssh or http://192.168.2.116/UI/ssh (Replace the IP with your own)

the system is based on the following CPU

CPU
processor       : 0
cpu             : APM82181
clock           : 800.000008MHz
revision        : 28.130 (pvr 12c4 1c82)
bogomips        : 1600.00
timebase        : 800000008
platform        : PowerPC 44x Platform
model           : amcc,apollo3g
Memory          : 256 MB

With that out of the way, A look at /etc/apt/sources.list revealed that it is a Debian Distro, the only problem with this is that debian stopped supporting this CPU some time ago, so you can’t go past Debian 8 (Jessie)

deb http://ftp.us.debian.org/debian/ squeeze main
deb http://ftp.us.debian.org/debian/ wheezy main
#deb-src http://ftp.us.debian.org/debian/ wheezy main
#deb http://ftp.us.debian.org/debian/ sid main

Checking the disk info with hdparm revealed that the disk is a WDC WD20EARX-00PASB0, which is as i expected a Caviar Green (SMR disk)

parted (The new fdisk so to speak) shows the following partition scheme for the existing system.

Model: ATA WDC WD20EARX-00P (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system     Name     Flags
 3      15.7MB  528MB   513MB   linux-swap(v1)  primary
 1      528MB   2576MB  2048MB  ext3            primary  raid
 2      2576MB  4624MB  2048MB  ext3            primary  raid
 4      4624MB  2000GB  1996GB  ext4            primary

And a “df -h” reveals

Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              1.9G  555M  1.3G  31% /
tmpfs                 5.0M     0  5.0M   0% /lib/init/rw
udev                   10M  6.7M  3.4M  67% /dev
tmpfs                 5.0M     0  5.0M   0% /dev/shm
tmpfs                 100M  4.6M   96M   5% /tmp
ramlog-tmpfs           20M  4.5M   16M  23% /var/log
/dev/sda4             1.9T  2.1G  1.9T   1% /DataVolume

A good alternative for this Gigabit Lan network attached storage might be openWRT, the same firmware I use for my routers !

there are things you need to know in advance though, first of which is that changing the firmware will require you to delete everything on the drive ! as Western Digital have used an unconventional bunch of things such as a 64 kB block size !

With that out of the way, you can skip down to the installing openWRT about the upgrade process step by step (Including backing up your system), then come back to why etc…

What if i want to revert back to the WD software ?

That is indeed a good question, and to make it easy to do that, I have already backed up the entire disk to another while I am sure that i don’t want to go back. Also worth mentioning that the latest firmware on the WD website dates back to 2015 ! which is at the time of writing 6 years ago !

Where can i find the up to date openWRT distribution for this drive ?

OpenWRT has a page dedicated to this drive, both the single and the Duo here (https://openwrt.org/toh/western_digital/mybooklive)

What are the benefits of the NAS box (enclosure), why not just take out the hard drive and put it in a PC somewhere.

The Western Digital My Book Live has a super low power CPU, and when the disk is spun down, it consumes very little energy (Not a significant load to your UPS for example), It is also fan-less, so it is with the exception of the spinning drive when it is spinning silent, which is also a nice thing, So i would argue that keeping it by updating it’s software is a good idea

Another reason is the amount of relevant software provided through openWRT packages, covering many more things than the original firmware (miniDLNA included).

How do i keep the system up to date

If you come from a debian background, you would normally apt-get update then apt-get upgrade and that is that, in OpenWRT, there is no such upgrade command, the upgrade command in openWRT is meant to upgrade 1 package specified by name, so the solution is the following line

 opkg list-upgradable | cut -f 1 -d ' ' | xargs -r opkg upgrade

WiFi for Arduino

Even though this looks like a long post, I have composed it for a friend and unlike mostly everything else on this blog, this is not just for my own reference, so it should be easy to follow and understand (I hope).

What for ?

This is a very valid question, Why would i use a slower Arduino and connect it to WiFi using an ESP8266 you ask, why not just use the ESP8266 or even ESP32 as both the WIFI and the microcontroller to run our code?
There are many situations where you would want to, the most common of which is the analogue and digital pins on an Arduino board, the friend I am writing this tutorial for is looking to use the 50 digital pins on an Arduino Mega Pro Embed as select lines for 50 Arduino pro mini boards, another might be the analogue pins on an Arduino (8 or 16 depending on the board), so digital and ADC pins on an Arduino might be needed.

You might ask why not an ESP32, it has a bunch of digital and analogue pins, the answer is that sometimes they are not enough, especially when you find out that the analogue pins on the ESP32 are divided into 2 groups, one of them is not usable if you enable WiFi.

Another valid reason is all the shields that have Arduino libraries but those libraries do not function with ESP, which is probably even more common of a problem than the pins problem.

So in short, even though the need might not arise very often, it does exist.

The ESP8266 as an Arduino WIFI shield

Arduino does not come with WiFi, there are shields from Arduino that provide WiFi, and those shields are based on ESP8266 which is a very cheap WiFi enabled microcontroller. but there is nothing stopping you from using any ESP8266 board and connecting it to your Arduino,

Which one: They should all work, and you probably already have one since you are here, I am personally using the slightly more expensive $4.6 boards that come with a USB-TTL chip and power regulator built in, if you want to use the cheaper boards (esp8266-01), you might want to connect it to the 3.3V output of your Arduino, but you will still need a level shifter, I would expect you also have a UART USB to serial board.

Price: models from the 01 ($2.5 each when you get 5 boards for $12 ) up to the 12E or 12F ($4.6 each when you buy them as 3 for $14). not bad for a WiFi enabled microcontroller !

Communication between Arduino and ESP8266

Arduino can talk to the shield either via UART or via SPI (Given the libraries written for this), SPI is up to three times faster than UART, but most of the time your application, be it sensor data or the like, will not be able to flood any of those 2 buses, In this post, I will cover both, SPI first then serial.

The components (hardware)

1- ESP8266 (Any variant should do)
2- Logic level shifter, since Arduino is 5V and ESPs are 3.3, I have been told that the ESP 12E and 12F are 5 volt logic tolerant, but I would think going with a logic shifter might save me something down the road, hours of debugging, or a new board, or something i fail to foresee
3- An Arduino, I am using a mega, but an UNO should do just fine (I will cover it)
4- Wires to connect all the above, and probably a breadboard (I like to solder things to a universal PCB board, but not everyone likes to do this)
5- A power supply, in my case a couple of micro USB cables and a 5V source that is my a power supply.

Software on the ESP8266

1:SPI: If you are going with SPI, you will need to flash JiriBilek / WiFiSpiESP onto your ESP8266, fortunately, this comes with an ino file that you can use your Arduino software to flash directly

2:UART-Serial: If you are going with serial, you might want to go with jeelabs / esp-link, mind you, Arduino themselves forked this before for their own WiFi shields, but since then, the jeelabs esp-link has added many features, so i would recommend you go with the original jeelabs.

Software on Arduino

1:SPI: if you have installed the SPI software from above on your ESP8266, the accompanying Arduino software would be JiriBilek / WiFiSpi, The library implements almost the same functions as the Arduino WiFi library.

2:UART-Serial: there is no library to go with this case that is beyond your regular serial bus if you want to exchange serial info, so if this is a 3D printer, software on your PC should be able to translate the data into serial, and it would be transparent, but what if you want to use WiFi from within Arduino, like a client that downloads pages or sends post data to pages,

Choice of UART-Serial vs SPI

UART-SERIAL, has certain advantages and disadvantages, with serial, i can simply update the software on the Arduino over the air over WiFi, I can get serial messages and use WiFi at the same time both as client and server, SPI on the other hand is faster, but it is not out of the box compatible with serial messages. Another disadvantage of SPI is that it needs a bit of extra code to allow the board to boot

Implementing WIFI over SPI

SPI – The hardware, how to connect

The H.SPI (On the ESP8266) is connected to the SPI on the Arduino like you would connect any SPI bus, with the addition of a logic level shifter (Red part in the photo), We connect Clock to clock, Slave select to select line, MOSI to MOSI and MISO to MISO, there is nothing to it. I have added a table for the Uno (Same for Arduino Pro Mini) and the Mega for your convenience

 NAME | ESP8266 | MEGA | Uno      | Logic Analyzer |
 SS   | D8      | D53  | D10      | CH0      | SS
 MOSI | D7      | D51  | D11      | CH1      | MOSI
 MISO | D6      | D50  | D12      | CH2      | MISO
 SCK  | D5      | D52  | D13      | CH3      | SCK

Now assuming you are done with the connection above, it is time to load some software.

SPI: Installing the WiFiSpiESP on the ESP8266

First, we need to load the software to ESP8266, the JiriBilek / WiFiSpiESP comes with a .ino file, so all you need to do is load that into Arduino studio, connect your esp8266, compile and upload, now this part is done, no modifications are needed to this code since all the control is passed on to the Arduino, compile and upload.

If you are having trouble uploading the code or selecting the board, my 12E board works in Arduino studio as NODEMCU V1.0, if you don’t have any ESP8266 boards in your boards list, you will need to add it, there are many tutorials on using Arduino with esp8266.

SPI: software on the Arduino

On the Arduino side, you will have to include the library (WiFiESP), then include it in your code, the library should be readily available in your libraries menu of your Arduino Studio.

NOTE: Both the library and the software you installed on your ESP need to have the same release number (0.2.5 at the time of writing) or it would not work, the software is hard coded not to work if they don’t match, you will be presented with the error (Protocol version mismatch. Please upgrade the firmware) in your serial console during runtime, I know this because a couple of weeks ago, I contacted the author (Jiri) through GitHub, and he brought both versions of the software and the library current so that they would match, it was a small thing but if you ever get this error in the future, you know where to go, he was quick to fix it within hours.

Now to the Arduino code, inside the library, there are examples, all you need to do is upload one of those examples, most likely, you would want to start off with the WiFiWebClient, this example that comes with the library needs to be modified in two locations, the first is the credentials to your WiFi, and the other is to change the server you are connecting to from www.example.com to wherever that web server is. this should get you started on most projects.

In my case, I have had to modify a few things in the script to make it work, first of all, a short delay needs to be inserted before we check if the WiFi is connected, the other is to not have it die but rather try again if it is not for a set number of times

WiFi using UART-Serial

UART-SERIAL should be the as easy, I should be back here

The ESP8266 has a TX and RX pin that should be connected in reverse to the ones on the Arduino, RX (Receive) should be connected to send, and send to receive, both boards need to share a common ground (reference voltage), and an Arduino mega should be able to provide 3.3 volts with sufficient current for the ESP8266 if you plan to power the ESP from the MEGA, if you have an ESP8266 with an onboard voltage regulator, you can simply add it to the power supply directly through the VIN pin (rather than the 3.3V pin)

Uploading jeelabs esp-link to the ESP8266

Start by downloading the zip file from GitHub,

Common name example.com is already present in a current certificate

Four days of godaddy SSL hell (starfield technologies certificate)

So, i am not writing this to mock godaddy or godaddy resellers or support, this is just a problem that you need to understand before you call godaddy (or any of their resellers) simply to save time and not to have to wait for 4 days like i did

When i submit my security signing request (csr file) to godaddy or wild west domains, the error i get reads

Common name example.com is already present in a current certificate.

The reason to this is that someone (probably you or a previous owner) already issued a certificate for that domain from another account.

SOLUTION: Certificate, or even expired certificate must be REVOKED, cancelled is not good enough, the magic word is REVOKED, when the certificate expires, you can not revoke it, you must contact support and tell them that you need to revoke it by email.

So, i have not taken the time to organize the text below this line yet, if you are arguing about something in an effort to reduce your wait time, see below for whatever you need, but again, i did not refine any text below this line or organized it or even checked that it is correct.

———————————————————————-

UPDATE: Godaddy wrong again, when i get the time i will listen to the recorded conversation (because my phone auto records all conversations) and tell you exactly what you need to do to not rely on the faulty godaddy manuals, in short this is what happened (as i remember it is close to this)

So, here is what my conversations with godaddy comes down to, not accurately, but in short, what it comes down to (for my reference, the file is godaddy ssl Voice-0003.amr)

But as i start to get skeptical about this resolving itself in a few hours, i will call jet (the very helpful customer care representative) again and see if anything can be done.

Godaddy (Jet): After canceling the certificate, you need to wait for three days
Me: No, i am sure we have to revoke it, and since it is expired, i can not revoke it
Godaddy (Jet): No you are mistaken, after cancelling, we wait for three days then put in a new request
Me: Ok i will wait
I wait for 2 days, then call again as my website is down
Me: are you sure that within 3 days the system will do cleanup, if the job runs once every three days, 2 days increases the odds of what i was saying being right, can you please double check ? my website has been down for two days
Godaddy: no need to check, there is nothing we can do
And after 3 days of still no luck, i call again
Me: hi, i have waited for 3 days
Godaddy (denis): yes sir, for a certificate to get cleared from the system it needs to be revoked, i will have them send you an email so we can revoke it by email.
me: Seriously, that’s what i said 3 days ago
Godaddy (denis): I wonder why they did not do that on the first day
Me: thanx anyways

Speed testing an internet connection

Well, there are a few ways to check the upload / Download speed of an internet connection, one way is speedtest.net which uses flash to download a file, and upload a file, both to a server close to you

On systems where we do not have a browser or do not have a browser that supports flash, one can download a file (With wget  on Linux for example), the quest would be this

You will need a file that is hosted on a network that you know for fact is faster than your own internet connection, for me, i have been using this one very successfully

cachefly.net 100mb.test

So, on a LINUX system, entering

wget http://cachefly.cachefly.net/100mb.test

On a casual 2.4Mb (That’s Mega Bit not Byte) , it should result in something like this

---------------------------------------------------------------------------------------
--2012-04-19 11:41:09--  http://cachefly.cachefly.net/100mb.test
Resolving cachefly.cachefly.net... 140.99.93.175
Connecting to cachefly.cachefly.net|140.99.93.175|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100mb.test'

 6% [=>                                     ] 6,897,290    284K/s  eta 5m 41s
---------------------------------------------------------------------------------------

While on a much faster connection i have somewhere else (theoretical 100Mb), the results are like this

--2012-04-19 08:44:20--  http://cachefly.cachefly.net/100mb.test
Resolving cachefly.cachefly.net... 140.99.93.175
Connecting to cachefly.cachefly.net|140.99.93.175|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104857600 (100M) [application/octet-stream]
Saving to: `100mb.test'

100%[======================================>] 104,857,600 41.2M/s   in 2.4s

2012-04-19 08:44:22 (41.2 MB/s) - `100mb.test' saved [104857600/104857600]

There are also other factors in internet connection speed that i will get to soon, for example, latency, and efficient routing.

things that i will get to when i have the time.