Superior in both Lossless compression, and Lossy compression, webp is the new image format by google

Already supported by all web browsers *(that i have tested it with), webP is indeed a promising format, so let us get to compressing our images

I have a big bunch of bitmaps that my scanner spits out (To avoid lossy jpeg compression the scanner’s driver produces), and i need them converted to lossless webp to save space (the first image I compressed went from 552MB bitmap to 183), that is 33% of the original size

So, under linux, this is how i would convert all BMPs into webp images, I think it is exectly the same on windows

on the command line, the command for compressing one image looks like

cwebp -lossless 00.bmp -o 00.webp

Now, the next step is to run them in a batch, copy the following text into a file and name it with the extension

On windows, you might have been using text editors that search or search&replace within files in a folder, one such tool i have used in windows is “source edit” by Joacim Andersson (Brixoft Software). that text editor does not seem to be maintained any longer as the developer seems to have moved into making games, but there are certainly many other editors that allow you to do the same thing.

On the other hand, on Linux, I don’t need to do that, the basic tools that come with the operating system allow for that, multi gigabyte files can be searched and have certain text replaced at the speed it takes to read them (Without having to open them for editing)

So, let us assume we have a folder with many text files (Including css or js or html or php files for example), to search that folder, we can combine

grep -Ril "text-to-find-here" /path/to/file/

-R (-r) look for files recursively
-l show file names, not the contents that were found
-i ….

Another tool which is better suited for looking in code is ack (ack-grep) which i will come back to cover in this article, and a newer tool that i have never used is

Now, replacing a string inside a file is simple, there is a cool tool called sed

sed -i '/TEXTTOFIND/ s//TEXTTOREPLACEWITH/g' verylargefile.txt

To tell tabs apart fast, you can give every terminal tab a name, just execute the following line inside that terminal window

echo -ne "\033]0;SOME TITLE HERE\007"

I am doing this on the default gnome terminal in Debian 11 (Bullseye), older methods no longer work

You will have to execute this every time in every SSH window to a remote machine, I am looking into a way to making the name permanent !

This is a simple task, and it is only simple because of foxlet (@FoxletFox on twitter)

Anyway, let us get to setting it up, to begin with, you don’t need to download MacOS, when using foxlet’s macOS-Simple-KVM, your virtual machine downloads MacOS on it’s own

Step 1: Make sure you have KVM ! and the relevant tools

apt-get install qemu-kvm libvirt-daemon qemu-system qemu-utils python3 python3-pip bridge-utils virtinst libvirt-daemon-system virt-manager

You know, the usual kvm setup ;), I am hoping you already have KVM, if not, see this post and install KVM first

Now that you have kvm, you need to insure that vhost_net is installed, loaded and enabled

modprobe vhost_net
lsmod | grep vhost

You will also need git to download macOS-Simple-KVM

git

git clone https://github.com/foxlet/macOS-Simple-KVM.git

Now, download MacOS base image that will download the rest of the operating system (catalina is the latest ?) options in that script are –high-sierra, –mojave, or –catalina.

./jumpstart.sh --catalina

The WD My Book Live is a NAS device based on Debian Linux, Since Debian stopped supporting this processor (APM82181), the device has received no updates and will probably never, so the next best thing to do in my opinion is to install openWRT.

Before you start

1- Only the first few paragraphs of this tutorial (STEPS 1 THROUGH 6) are the instructions you need, the remaining is just for extra reference and in short you don’t need to read it to have your device running, but I do recommend YOU SKIM THE WHOLE THING BEFORE YOU START.
2- This procedure requires you to take the disk out and install it on a PC to switch the firmware, then put it back
3- The upgrade will delete all your data, You will need to move your data that is already on your WD NAS drive somewhere else while the upgrade is ready.

Step 1: Move any existing data BEFORE TAKING APART.

Move any data you may have on the drive to a temporary location outside the NAS drive. this has to be done before taking the drive apart as the unconventional 64 kB block size of the disk will be nothing but trouble if you want to extract the data while mounting the disk to a linux PC for example.

Step 2: Take the disk apart

I have included photos to help you do that, it is not rocket science.

Step 3: Mount the disk on a linux PC (Windows and MAC should work)

and mount it to a linux PC (Windows might work with software such as etcher, but i have no guarantees).

Step 4: Download the openWRT firmware

Go to the drive’s page on the openwrt website (Here), and download it to your Linux (Or windows) PC

Step 5: Write the firmware to the disk.

Decompress the file, then copy it to the drive with a command similar to the command below, but make 100% sure to replace sdx with your own drive designation

 dd if=/root/wdsata.img of=/dev/sdx bs=64k

Write the firmware to the disk, overwriting it, and effectively loosing any data you did not backup in step 1

Step 6: Put the drive back in the enclosure

Nothing to say here, this is the reverse of step 2

Once it is in the enclosure, you can not just connect it to your router as it in itself has this port defined as 192.168.1.1 and is serving dhcp !

Step 8: Create the data partition

At this stage, your device will boot, but you will need to create/expand the data partition, the partition that should not be overwritten when you upgrade the firmware for example.

opkg update
opkg install gdisk blkid openssh-sftp-server
gdisk -i /dev/sda

As soon as gdisk opens, you may be presented with the following message, if so

Found valid MBR and corrupt GPT. Which do you want to use? (Using the
GPT MAY permit recovery of GPT data.)
 1 - MBR
 2 - GPT
 3 - Create blank GPT

Chose 1 to maintain the 2 partitions we have, Now hit the command (w) to write and confirm, then quit, gdisk has just switched your disk to GPT from MBR, now run gdisk again the same way (gdisk -i /dev/sda)

n for new partition, accept the (3) for partition number, use the number (2097152) for alignment with 4K sector advanced format nearest to the 1GB mark

mkfs.ext4 /dev/sda3
mkdir /share
blkid /dev/sda3

in my case, the UUID was as follows UUID="9643bd00-f117-4074-a252-7ea30a5174e2" yours will certainly be different, so in my fstab i added the following lines near the end

config mount
option target '/share'
option uuid '9643bd00-f117-4074-a252-7ea30a5174e2'
option enabled '1'

Now, network sharing is what i was originally interested in when i got this unit, and it is why I am replacing it’s firmware, so to installing samba

opkg update && opkg install samba4-server luci-app-samba4

Now, add the following line to /etc/passwd to add me as a user to the system

yazeed:*:1000:65534:yazeed:/var:/bin/false

Now, run the command

passwd yazeed
smbpasswd -a yazeed

You are done.

FAQ

Is the hardware and the new openWRT firmware compatible with my 8TB hard drive

Yes it is, I have found many people asking if the hardware supports drives over 2TB, the answer is yes, but you will have to use the GPT rather than the MBR (See steps above)

about the original firmware

What is that vulnerability about

it comes from WDs cloud service, bottom line is that many devices were completely wiped remotely by malicious users and it is unknown if the data itself leaked, so yes, it is very serious

What is the difference between quick factory restore and full factory restore

Quick factory restore is probably what you are looking for, the later seems to do a zero fill on the hard drive after performing a factory restore to disallow data retrieval (For example before you sell it), you can verify this by logging in using SSH, and by the fact that the tool tips state something to that effect.

Inspecting the device

To begin with, I logged in via SSH and inspected some stuff, to enable SSH access on the My Book Live original firmware, you will need to visit a page at a URL such as http://mybooklive/UI/ssh or http://192.168.2.116/UI/ssh (Replace the IP with your own)

the system is based on the following CPU

CPU
processor       : 0
cpu             : APM82181
clock           : 800.000008MHz
revision        : 28.130 (pvr 12c4 1c82)
bogomips        : 1600.00
timebase        : 800000008
platform        : PowerPC 44x Platform
model           : amcc,apollo3g
Memory          : 256 MB

With that out of the way, A look at /etc/apt/sources.list revealed that it is a Debian Distro, the only problem with this is that debian stopped supporting this CPU some time ago, so you can’t go past Debian 8 (Jessie)

deb http://ftp.us.debian.org/debian/ squeeze main
deb http://ftp.us.debian.org/debian/ wheezy main
#deb-src http://ftp.us.debian.org/debian/ wheezy main
#deb http://ftp.us.debian.org/debian/ sid main

Checking the disk info with hdparm revealed that the disk is a WDC WD20EARX-00PASB0, which is as i expected a Caviar Green (SMR disk)

parted (The new fdisk so to speak) shows the following partition scheme for the existing system.

Model: ATA WDC WD20EARX-00P (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system     Name     Flags
 3      15.7MB  528MB   513MB   linux-swap(v1)  primary
 1      528MB   2576MB  2048MB  ext3            primary  raid
 2      2576MB  4624MB  2048MB  ext3            primary  raid
 4      4624MB  2000GB  1996GB  ext4            primary

And a “df -h” reveals

Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              1.9G  555M  1.3G  31% /
tmpfs                 5.0M     0  5.0M   0% /lib/init/rw
udev                   10M  6.7M  3.4M  67% /dev
tmpfs                 5.0M     0  5.0M   0% /dev/shm
tmpfs                 100M  4.6M   96M   5% /tmp
ramlog-tmpfs           20M  4.5M   16M  23% /var/log
/dev/sda4             1.9T  2.1G  1.9T   1% /DataVolume

A good alternative for this Gigabit Lan network attached storage might be openWRT, the same firmware I use for my routers !

there are things you need to know in advance though, first of which is that changing the firmware will require you to delete everything on the drive ! as Western Digital have used an unconventional bunch of things such as a 64 kB block size !

With that out of the way, you can skip down to the installing openWRT about the upgrade process step by step (Including backing up your system), then come back to why etc…

What if i want to revert back to the WD software ?

That is indeed a good question, and to make it easy to do that, I have already backed up the entire disk to another while I am sure that i don’t want to go back. Also worth mentioning that the latest firmware on the WD website dates back to 2015 ! which is at the time of writing 6 years ago !

Where can i find the up to date openWRT distribution for this drive ?

OpenWRT has a page dedicated to this drive, both the single and the Duo here (https://openwrt.org/toh/western_digital/mybooklive)

What are the benefits of the NAS box (enclosure), why not just take out the hard drive and put it in a PC somewhere.

The Western Digital My Book Live has a super low power CPU, and when the disk is spun down, it consumes very little energy (Not a significant load to your UPS for example), It is also fan-less, so it is with the exception of the spinning drive when it is spinning silent, which is also a nice thing, So i would argue that keeping it by updating it’s software is a good idea

Another reason is the amount of relevant software provided through openWRT packages, covering many more things than the original firmware (miniDLNA included).

How do i keep the system up to date

If you come from a debian background, you would normally apt-get update then apt-get upgrade and that is that, in OpenWRT, there is no such upgrade command, the upgrade command in openWRT is meant to upgrade 1 package specified by name, so the solution is the following line

 opkg list-upgradable | cut -f 1 -d ' ' | xargs -r opkg upgrade

The new way of mounting a disk image created with dd, dd_rescue, or ddrescue has become much simpler than before, all you need now is to issue the command

losetup --partscan --find --show disk.img

then above will tell you what loop device is being used, let us assume it is /dev/loop0, right after, a quick fdisk -l should show you the partitions, in my case, i have /dev/loop0p1 and /dev/loop0p2

mount /dev/loop0p1 /mnt

now, the first partition is mounted, to reverse this, you will need to first unmount /mnt then, you can delete the loop device with

losetup -d /dev/loop0

At this stage you can mount it like any other device, in read-write mode by default, if you want to mount it in read only mode, you can use the -o switch

sudo mount -o ro /dev/loop0px /hds/loopdevice

---

Now, if you have DDd a partition rather than a block device (disk) and want to mount it, you can simply mount it as a loop device, and then mount the loop device (loop0) without a Px at the end

The setup assumed in this post is as follows, you are working on a remote windows computer, there is a Linux KVM host computer running guest virtual machines somewhere (OS of guest irrelevant), and you would like to connect to a guest machine’s console (which may be running windows, linux, macOS, or any operating system)

KVM, by default only allows people to connect through VNC to the console of a virtual machine if they are using the local host computer, so here are the tips on creating a tunnel to the host computer and connecting to your KVM virtual machine.

Windows does not support VNC very well, (Most VNC servers don’t run well on windows), but the VNC server here is not windows, it is KVM that is providing the VNC server to the guest’s console.

1- Create a tunnel (Putty on windows), simply put, save the connection in putty to that host machine, then under tunnels you will need to have something like this

Just create a tunnel for port 5900 and the destination localhost:5900 (5901 for the second virtual machine and so on), leave all other tunnel options unchecked/default

2- to know which ones are enabled on your machine run this command

netstat -tlpn | grep 590

3- VNC should now connect to localhost:9500 for example (I am using tightVNC on windows), and that connection should be automatically router to the KVM host, which will display the guest’s console depending on the port (every guest has it’s own port)

By default, KVM gives your virtual machine a realtek rtl8139 Ethernet adapter, with an ancient 100Mbit/Second speed, we all need gigabit Ethernet adapter for the KVM guest.

The answer is changing the string rtl8139 with virtio in the XML file of the virtual machine, then installing the drivers

The steps i use are

Run virtual machine with the realtek adapter to download the other adapter’s driver
once the adapter is there, shutdown the virtual machine guest (Windows guest), then edit the xml of the guest, and restart libvirtd
start the KVM guest again
open with VNC, start the device manager, install the driver you downloaded.

You are good, the adapter should report the speed of 10Gbit/second (10 gigabit per second)

One annoying thing is that all windows drivers come in a big ISO file, you probably just want the driver you need.

I will add the download links in the coming few days, but you can get them right now if you like from fedora, the fedora windows guest drivers should work on any linux distribution (Debian, ubuntu, etc…)

Well, we all know FTP, HTTP and other protocols would do the trick, but they are not the fastest way. because of setup and other considerations, especially when we are planing to use both windows and linux in the transactions, a low overhead method would be samba, or windows network file sharing

In windows, it is simple, we all know how to share a folder, and we all know how to open a shared folder over the network, in linux it is just as simple.

The detailed explanation of how to setup the samba server and share a folder is already in a post, but from the client side, you can copy a file by simply mounting the shared drive onto the linux server and copy or move files like you would a normal file or folder.

To mount we can simply execute the following

1- Install the client tools to mount the folder
apt-get install cifs-utils

then
mkdir /hds
mkdir /hds/smbmount

mount -t cifs //192.168.15.116/sharedfolder /hds/smbmount -o username=techg,noexec

That’s it, now use CP and MV as you would normally

One problem i faced with this method before was that my WD My Book Live shares would not mount, it threw an error

mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

The logs did not provide much help either

CIFS VFS: cifs_mount failed w/return code = -22

Turns out that the WD MyBook Live has an older version of Debian, and samba on that device had a max-v of 2, so the solution is to connect while specifying which version of samba works

mount -t cifs //192.168.2.116/sharedfolder /hds/smbmount -o vers=2.0,username=techg,noexec

The values for Version can be 2.0 2.1 3.0

Being a linux super user / server administrator for many many years, i still use windows for my work / personal use, working with putty all the time to access the linx machines.

So just in order to drop the laptop, and replace it with a flash stick in my pocket, and maybe sometimes an external hard drive (when i am dealing with big files), I have decide to drop windows all together, in favor of a linux installation on a flash stick in my pocket.

Then, i can use the work PC, and the home PC, and my laptop anywhere without having multiple installations of the same thing.

Because my computer has data aging back to 2001 that i sometimes need or use etc… I will probably be keeping my windows installation in my laptop even when i boot it from a flash stick with Debian Linux on it.

But for this ambitious project, i have to compile a list of programs that replace my everyday use programs, i am not talking adobe premier here, i am talking more like adobe photoshop. and something to replace adobe dreamweaver.

To do this, i will use a USB3 64GB flash stick, and dedicate 32GBs to a FAT/NTFS system suitable for use on any windows PC, and a bootable EXT4 (has to be the second one because windows only sees the first partition on flash sticks) that boots debian wheezy. I have left a post somewhere on this blog on how to do that that i will link from here very soon.

So, here is a list of things i use every day, and candidates for linux, I will keep growing this list as i go because i think it is very essential

	Windows	Linux
I use it for all my image processing needs	Adobe Photoshop	The Gimp
I might use it for documents i get by email etc…	Microsoft office	One of the 2 popular office suites for Linux.
I use dreamweaver for everything, All the formatted documents i create for myself or to send to others are either HTML or PDF, plain text documents are the ones i use most of the time, but that comes with vanilla debian	Adobe dramweaver	Quanta Plus, Aptana, KompoZer, BlueGriffon
Web browser	Firefox	Whatever name the chose for Firefox
	WinSCP
	FileZilla
	VMWARE workstation	VMWARE workstation
	Windows Network	Samba