gigabit Ethernet VirtIO driver for Windows 10 64bit

By default, KVM gives your virtual machine a realtek rtl8139 Ethernet adapter, with an ancient 100Mbit/Second speed, we all need gigabit Ethernet adapter for the KVM guest.

The answer is changing the string rtl8139 with virtio in the XML file of the virtual machine, then installing the drivers

The steps i use are

Run virtual machine with the realtek adapter to download the other adapter’s driver
once the adapter is there, shutdown the virtual machine guest (Windows guest), then edit the xml of the guest, and restart libvirtd
start the KVM guest again
open with VNC, start the device manager, install the driver you downloaded.

You are good, the adapter should report the speed of 10Gbit/second (10 gigabit per second)

One annoying thing is that all windows drivers come in a big ISO file, you probably just want the driver you need.

I will add the download links in the coming few days, but you can get them right now if you like from fedora, the fedora windows guest drivers should work on any linux distribution (Debian, ubuntu, etc…)

Moving files in linux with samba

Well, we all know FTP, HTTP and other protocols would do the trick, but they are not the fastest way. because of setup and other considerations, especially when we are planing to use both windows and linux in the transactions, a low overhead method would be samba, or windows network file sharing

In windows, it is simple, we all know how to share a folder, and we all know how to open a shared folder over the network, in linux it is just as simple.

The detailed explanation of how to setup the samba server and share a folder is already in a post, but from the client side, you can copy a file by simply mounting the shared drive onto the linux server and copy or move files like you would a normal file or folder.

To mount we can simply execute the following

1- Install the client tools to mount the folder
apt-get install cifs-utils

then
mkdir /hds
mkdir /hds/smbmount

mount -t cifs //192.168.15.116/sharedfolder /hds/smbmount -o username=techg,noexec

That’s it, now use CP and MV as you would normally

One problem i faced with this method before was that my WD My Book Live shares would not mount, it threw an error

mount error(22): Invalid argument
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

The logs did not provide much help either

CIFS VFS: cifs_mount failed w/return code = -22

Turns out that the WD MyBook Live has an older version of Debian, and samba on that device had a max-v of 2, so the solution is to connect while specifying which version of samba works

mount -t cifs //192.168.2.116/sharedfolder /hds/smbmount -o vers=2.0,username=techg,noexec

The values for Version can be 2.0 2.1 3.0

Switching from windows to flash stick linux, my whole experience

Being a linux super user / server administrator for many many years, i still use windows for my work / personal use, working with putty all the time to access the linx machines.

So just in order to drop the laptop, and replace it with a flash stick in my pocket, and maybe sometimes an external hard drive (when i am dealing with big files), I have decide to drop windows all together, in favor of a linux installation on a flash stick in my pocket.

Then, i can use the work PC, and the home PC, and my laptop anywhere without having multiple installations of the same thing.

Because my computer has data aging back to 2001 that i sometimes need or use etc… I will probably be keeping my windows installation in my laptop even when i boot it from a flash stick with Debian Linux on it.

But for this ambitious project, i have to compile a list of programs that replace my everyday use programs, i am not talking adobe premier here, i am talking more like adobe photoshop. and something to replace adobe dreamweaver.

To do this, i will use a USB3 64GB flash stick, and dedicate 32GBs to a FAT/NTFS system suitable for use on any windows PC, and a bootable EXT4 (has to be the second one because windows only sees the first partition on flash sticks) that boots debian wheezy. I have left a post somewhere on this blog on how to do that that i will link from here very soon.

So, here is a list of things i use every day, and candidates for linux, I will keep growing this list as i go because i think it is very essential

Windows Linux
I use it for all my image processing needs Adobe Photoshop The Gimp
I might use it for documents i get by email etc… Microsoft office One of the 2 popular office suites for Linux.
I use dreamweaver for everything, All the formatted documents i create for myself or to send to others are either HTML or PDF, plain text documents are the ones i use most of the time, but that comes with vanilla debian Adobe dramweaver Quanta Plus, Aptana, KompoZer, BlueGriffon
Web browser Firefox Whatever name the chose for Firefox
WinSCP
FileZilla
VMWARE workstation VMWARE workstation
Windows Network Samba

 

DD_RESCUE ( GDDRESCUE’s ddrescue ) for disks with Advanced Format ( AF ) 4KiB sectors 4096 byte

1- Before using dd, ddrescue, or dd_rescue, you need to know which disk is which, you can do that by simply using the command “fdisk -l” in my case, the old disk turned out to be /dev/sdb and the new un-partitioned disk is /dev/sdc.

So, i have been cloning a 2TB hard drive ( WD20EARS ) to a WD20EARX, same disk, but with a few differences

WD20EARS is sata 2 and the other is sata 3, another difference is that using “hdparm -I /dev/sdb” the older WD20EARS reports (And should not be true)

WD20EARS

Logical/Physical Sector size:           512 bytes

wile with “hdparm -I /dev/sdc” the newer WD20EARX reports

        Logical  Sector size:                   512 bytes
        Physical Sector size:                  4096 bytes
        Logical Sector-0 offset:                  0 bytes

The first clone did not work for a reason unknown to me, i cloned my NTFS disk with ddrescue (gddrescue) on a linux (because i don’t know how to clone on windows) and then plugged it into windows, where it simply did not work, and in disk management reported the disk as un-partitioned space, so now i want to do the thing again, but i don’t want that slow performance, so i increased block size to 4KiB. (UPDATE: THE NEW COPY WITH 4KiB DID WORK BUT I DONT KNOW IF THE 4KiB SIZE IS RELEVANT, MAYBE YOU SHOULD TAKE A LOOK AT THE SECOND DIFFERENCE BETWEEN THE DISKS UP AT THE BEGINNING OF THE POST)

For now, i will try the cloning with the command (Only change the block level for advanced format hard drives)

ddrescue --block-size=4KiB /dev/sdb /dev/sdc rescue2.log

And if all of your data is important, you can ask ddrescue to retry every bad block 3 times (or as many times as you wish) with the -r command

ddrescue --block-size=4KiB -r3 /dev/sdb /dev/sdc rescue2.log

And what do you know, the disk now works on my WINDOWS machine 😀 no errors and no nothing, great, so now to some details about the copy

The result up to now is that i am reading at a maximum of 129MB while the average (in the first 60 GBs is 93018 kB/s), if this continues, i will be done in less than 6 hours.

The part that does not make any sense to me is that western digital states clearly in the specs that the maximum (Host to/from drive (sustained)) is 110 MB/s for both drives, it must be that i need to wait a bit more and see what that actually means.

rescued:         0 B,  errsize:       0 B,  errors:       0
Current status
rescued:    74787 MB,  errsize:       0 B,  current rate:     119 MB/s
   ipos:    74787 MB,   errors:       0,    average rate:   93018 kB/s
   opos:    74787 MB,     time from last successful read:       0 s
Copying non-tried blocks...

Now, once done, you can have the OS reload the partition table without having to restart, you can simply use the command partprobe

partprobe
or
partprobe /dev/sdc

To use partprobe, you need to install parted

apt-get install parted

If it were a linux drive, an advanced format drive would not have it’s first sector on sector 63 but rather on sector 2048, which is at exactly 2KiB, it could (but usually does not) start at any other value divisible by 8.

Windows probably does something similar for our AF Disk, so asking parted about our ntfs disk, this is what parted says

Model: ATA WDC WD20EARS-00M (scsi)
Disk /dev/sdb: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  2000GB  2000GB  primary  ntfs

1049kB is 1074176 bytes, Which when divided by 8 is 134272 (divisible by 8).

NOTES:
-There is a tool specifically for cloning ntfs volumes called ntfsclone, i am not sure what extra features it provides that are specific to ntfs, i have never used it before, with my disk that has bad blocks, i can only rely on gddrescue.
-A block is 512 on regular drives, and 4096 on newer ones, if you want to backup the hard drive’s geometry, you can do one of the following
Backup the first 63 blocks (MBR + Bootloader). on a “non advanced format” drive

dd if=/dev/sda of=/mnt/storage/sda.vbr bs=512 count=63

On an advanced format drive, we can try

dd if=/dev/sda of=/mnt/storage/sda.vbr bs=4096 count=63

Which, will make us read 258048 bytes rather than the traditional 32256 bytes (around 250K rather than 32K)

WINSCP for linux !

An application for windows that i wantin Linux is WINSCP, but it seems the author of WINSCP says (on his forum) that ” Sorry, there’s no chance for that.”

In any case, i have no doubts there are hundreds of applications that can do the Job, in fact the file browser that comes with your gnome or KDE already opens FTP and SFTP and SCP connections, so you need to look no further.

There are also applications that can mount a remote file system that is run on SSH, xxx is one such software

But truth be told since the days of Norton commander, i have always liked the two window view that winSCP is similar to.

So in this post, i will add screenshots of the applications similar to WINSCP, i will try both krusader and filezilla (Yes, filezilla does support)

apt-get update

then

apt-get install krusader filezilla

With Krusader, it is a good idea to install 
apt-get install kdiff3 kompare xxdiff krename rar unrar zip

Another software to be tested would be snowflake, confusingly it is being renamed to muon, which is already the name of a package manager for debian !, in any case, installing snowflake is as simple as downloading the deb file then installing it

wget https://github.com/subhra74/snowflake/releases/download/v1.0.4/snowflake-1.0.4-setup-amd64.deb

then install it

dpkg -i snowflake-1.0.4-setup-amd64.deb

Worth noting that on my 4K display which has a 200% setting, snowflake is not usable, the font is so small, and clicking on something is a challenge, so to work around this while the maintainers fix this for people who have settings like mine, i run snowflake from the terminal with

java -Dsun.java2d.uiScale=2.5 -jar /opt/snowflake/snowflake.jar

And now we have both, on my computer which is a fresh install, krusader was a 90MB download, in your case, it is probably much less because most of the things downloaded are libraries you probably already have.

in any case, let me take those screenshots of WINSCP’s alternatives and get back to this post

Linux File System commands ext2, ext3, ext4

UPDATE: This post has been added at easywebdns : recovering ext3 hard disks

So, here are the tools you need

1- Checking a linux partition for bad sectors / Bad Blocks without deleting the data

badblocks -o /path/somefiletowriteto.txt /dev/sdb1

this will do a check and write the bad block numbers to a text file.

Please note that 6TB and 8TB hard drives will not work out of the box with badblocks program, you will probably get a message informing you that the boundary must fit a 32bit integer (Value too large for defined data type), in that case, you can always increase the block size with the (-b 4096) switch, this will give you 8 times more block addressing space (Since the default is 512 bytes.

If you want it to test the disks hard but you are OK with deleting the data (Say after you dd_rescue), then try the -w switch

badblocks -w -o /path/somefiletowriteto.txt /dev/sdb1

Or, to see information while it scans

badblocks -wsv /dev/sdd > /root/250bad-blocks.txt

Notice that the -w has to come before the -o since the -o must be followed by a file name.

2- fsck /dev/sdb1 , the all famous, need i say more ?

3- tune2fs

tune2fs -O ^has_journal /dev/sdb1

remove the Journal (Converting ext3 to ext2 file system)

4- Mounting a disk: mount -t ext3 /dev/sdb1 /adirectory

5- debugfs <- use it in Read only more, this thing is like brain surgery, 1 mistake and your FS is playing chess in heaven with all the other dead disks

6- dd_rescue /dev/sdb1 /directory/mydisk.img

7- Running fsck on a disk image

fsck -y /path/mydisk.img

8- Mount the image

mount /pathto/backup.img /mountpoint/data

this is a nice link for you http://www.linuxjournal.com/article/193 that already has info about the stuff i mentioned here

Those are probably the most common, will add more when i remember them

Bruit force attacks and hacking my web server

My web server got hacked today, i know because my datacenter contacted me today telling me that there is a bruit force attack originating from my server to another server on a different network, so what is happening is that my server got hacked, then the hacker is using the server she hacked to hack other servers by sending FTP requests.

So, how come i got hacked when i am so obsessed with security, well, in reality, this is just an intermediate machine that i used to run a certain script that would move my mail server, and i did not (and did not see the need) to secure it.

What i usually do to secure my server is simply install fail2ban, in this case i did not out of lazyness but here is how i got hacked and how fail2ban would have protected me.

Before i show you the log files, this whole problem would not happen if i had a strong password combined with fail2ban

In the complaining partie’s log files

Tue Jul 24 22:28:27 2012: user: hauvouuc service: ftp target: yyy.yyy.yyy.yyy source: xxx.xxx.xxx.xxx
Tue Jul 24 22:28:27 2012: user: pkmcndgq service: ftp target: yyy.yyy.yyy.yyy source: xxx.xxx.xxx.xxx
Tue Jul 24 22:28:27 2012: user: malumdvc1 service: ftp target: yyy.yyy.yyy.yyy source: xxx.xxx.xxx.xxx

In my log files (auth.log):

Many lines like the following right below each other

Jul 24 18:03:08 run sshd[14229]: pam_unix(sshd:auth): check pass; user unknown
Jul 24 18:03:08 run sshd[14229]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=9.12-14-84.ripe.coltfrance.com 
Jul 24 18:03:10 run sshd[14229]: Failed password for invalid user ts3 from 84.14.12.9 port 41014 ssh2
Jul 24 18:03:11 run sshd[14231]: Invalid user ts3 from 84.14.12.9

Anod some lines like this

Jul 25 15:30:46 run sshd[10728]: pam_unix(sshd:auth): check pass; user unknown
Jul 25 15:30:46 run sshd[10728]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=217.119.29.135 
Jul 25 15:30:48 run sshd[10728]: Failed password for invalid user public from 217.119.29.135 port 34292 ssh2
Jul 25 15:30:48 run sshd[10730]: Address 217.119.29.135 maps to gamma2-7.cust.smartspb.net, but this does not map back to the address - POSSIBLE BREAK-IN ATTEMPT!
Jul 25 15:30:48 run sshd[10730]: Invalid user public from 217.119.29.135

Thousands of lines like this one

Jul 24 14:12:38 run sshd[2025]: error: connect_to 213.186.33.207 port 80: failed.
Jul 24 14:12:39 run sshd[2025]: error: connect_to 192.168.10.24 port 2110: failed.
Jul 24 14:12:39 run sshd[2025]: error: connect_to 195.130.65.50 port 80: failed.

OR

Jul 24 06:41:19 run sshd[9824]: error: connect_to 213.186.33.207 port 80: failed.
Jul 24 06:41:19 run sshd[13434]: Failed password for invalid user test from 202.28.123.191 port 37830 ssh2
Jul 24 06:41:20 run sshd[9824]: error: connect_to 213.186.33.207 port 80: failed.

And more like this

Jul 24 08:19:18 run sshd[20882]: pam_unix(sshd:auth): check pass; user unknown
Jul 24 08:19:18 run sshd[20882]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=puck748.server4you.de 
Jul 24 08:19:21 run sshd[20882]: Failed password for invalid user kk from 85.25.235.73 port 49213 ssh2
Jul 24 08:19:21 run sshd[20884]: Invalid user css from 85.25.235.73

Installing my 3TB hard drive on Debian linux step by step

It is simple, here is what you need to know

You can format it EXT4, but ext2 and ext3 are also OK ! ext2 and ext3 allow up to 16TB disks, and file sizes of up to 2TB, ext4 allows much more.

Any linux kernel newer than 2.6.31 should work just fine with “Advanced format” drives using the exact same steps in this article.

MBR only supports 2TB drives, you need GPT, so let us get started

1- apt-get update
2- apt get install parted
3- parted /dev/sdc
4- mklabel gpt
5- Answer yes to: Warning: The existing disk label on /dev/sdb will be destroyed and all data on this disk will be lost. Do you want to continue?
Yes/No? yes
6- mkpart primary ext4 0% 100% (to make a partition as big as the disk (will occupy starting from first megabyte (for alignment) to the end of disk))
7- quit

FYI, if you want multiple partitions, here are the 2 lines that should replace step 6
6- mkpart primary ext4 0% 40%
6- mkpart primary ext4 40% 100%

and remember to format both (sdc1 and sdc2) when you are done with parted

Now to formatting the drive

mkfs.ext4 /dev/sdc1

Before mounting it, i like ext4, but i don’t want a journaling OS on this drive that is not the system drive, so i will need do a few things to the drive first

Lazy writeback
tune2fs -o journal_data_writeback /dev/sdc1

No Journaling
tune2fs -O ^has_journal /dev/sdc1

Now to check what we have

dumpe2fs /dev/sdc1 |grep ‘Filesystem features’
Or maybe if you want the whole thing on the screen
dumpe2fs /dev/sdc1 |more

if has_journal option exist when executing the first – you have journal on the file system

And there we are, Now we need to mount it at boot time by adding it to fstab, to do that, we will need the disk’s unique ID !

8- Now executing the following command will give you the unique ID of this new partition for use with fstab (The disk list we will edit below in step 10)
blkid /dev/sdc1
9- create the directory where you want to mount your hard disk, for example
mkdir /hds
mkdir /hds/3tb
10- Now, we add the following line to fstab, notice that noatime increases performance, but some applications might need or rely on it. postfix does not and i have verified that.

UUID=b7a491b1-a690-468f-882f-fbb4ac0a3b53       /hds/3tb            ext4     defaults,noatime                0       1

11- Now execute
mount -a

You are done,. if you execute
df -h
You should see your 2+TB hard drive in there !

To make sure the drive is aligned correctly, i like to write a file on it and see how fast that goes… so let us use a 2GB file

dd if=/dev/zero of=/hds/WD2000_3/deleteme.img bs=1M count=2000

Outcome came out (for a western digital black 2TB)
First run: 2097152000 bytes (2.1 GB) copied, 5.94739 s, 353 MB/s
Consecutive runs: 2097152000 bytes (2.1 GB) copied, 11.1405 s, 188 MB/s
Outcome came out for a western digital green 3TB
First run: 2097152000 bytes (2.1 GB) copied, 8.32337 s, 252 MB/s
Consecutive runs: 2097152000 bytes (2.1 GB) copied, 14.376 s, 146 MB/s

the consecutive runs give close results, what i printed here is the average

Broadcom wireless with Debian Squeeze / Wheezy

My old tablet (HP tc4200) had problems with the wireless adpater , A broadcom BCM4309

To find out what the Broadcom wireless adapter model is i issued

lspci -vvnn | grep 14e4

For yours, you may need to check with this website here as you may or may not need the sta or the b43legacy driver, in general here are the popular models

STA – BCM4311, BCM4312, BCM4313, BCM4321, BCM4322, BCM43224, BCM43225, **BCM43227, **BCM43228

b43 – BCM4306/3, BCM4311, BCM4312, BCM4318, BCM4320

b43legacy – BCM4301, BCM4306, BCM4306/2

http://www.linuxwireless.org/en/users/Drivers/b43#Supported_devices

The, now that i know… i edited /etc/apt/sources.list and added the contrib and non-free repositories

then

apt-get update

apt-get install firmware-b43-installer b43-fwcutter

And what do you know, just reboot and it works

Rescueing data of a failed hard drive

I accedientally pulled the power plug of a PC from the socket, and that PC was just starting to boot. The seagate hard drive inside stopped working, and the bad sectors turned out to affect the partition table, in any case, i slaved it on a windows vista PC, then into the Computer management, disk management panel, and what do you know, as if it has no paritions…

The solution to detect the boundaries of the 4 partitions it had is software called XXXX

Ran the software (The analyze option) , and what do you know, my partions exactly, 100MB made by Windows 7, a 479 GB partition for Windows, a 1GB swap partition for linux and an EXT3 partition for Linux…

So happily i asked the software to write the partitioning info to the disk, but the disk won’t hold the data, the bad sectors are where Windows writes the partition information

So, i ran down to the computer shop (In our building), and got the same exact drive (Seagate 500GB Model number xxx)

Mounted both on a Linux machine as slaves, both the damaged and the target.

To find out which one is SDC and which one is SDB, i watched as the linux machine booted, and as it booted, it threw in errors saying SDB all the time, so i know that SDB is the busted drive !

Installed gddrescue (apt-get install gddrescue), and ran it with the following command

ddrescue /dev/sdb /dev/sdc resumelog.log

(The additional log file helps us resume in case of interruption)

Once that is done, i put the new hard drive in a Windows machine, still can not see any partition info

1- Ran xxxx, it can see the 4 partitions, write changes… and what do you know, the partitions stick, we are good to go, i restart, but still, Windows can now see the partitions, but thinks drive G is not formated !

So i opened the command prompt (Elevated), then ran the command

chkdsk g: /f

the /f stands for fix, the thing took some time, but after the restart drive G works fine, all files are in there, and no one wants to kill me no more 🙂