UNIX Monitoring pipe activity with pv

Monitoring pipe activity with pv

$ dd if=/dev/zero | pv > foo

522MB 0:00:06 [ 109MB/s] [ <=> ]

When pv is added to the pipeline, you get a continuous display of the amount of data that is being transferred between two pipe endpoints. I really dig this utility, and I am stoked that I found the catonmat website! Niiiiiiiiice!

UPDATE:

Try using pv when sending stuff over the network using dd. Neato.

[root@machine2 ~]# ssh machine1 “dd if=/dev/VolGroup00/domU2migrate”|pv -s 8G -petr|dd of=/dev/xen02vg/domU2migrate
0:00:30 [11.2MB/s] [====> ] 4% ETA :10:13

Want to rate limit the transfer so you don’t flood the pipe?

-L RATE, –rate-limit RATE
Limit the transfer to a maximum of RATE bytes per second. A suffix of “k”, “m”, “g”, or “t” can be added to denote kilobytes (*1024), megabytes, and so on.

-B BYTES, –buffer-size BYTES
Use a transfer buffer size of BYTES bytes. A suffix of “k”, “m”, “g”, or “t” can be added to denote kilobytes (*1024), megabytes, and so on. The default buffer size is the block size of the input file’s filesystem multiplied by 32 (512kb max), or 400kb if the block size cannot be determined.

Already have a transfer in progress and want to rate limit it without restarting?

-R PID, –remote PID
If PID is an instance of pv that is already running, -R PID will cause that instance to act as though it had been given this instance’s command line instead. For example, if pv -L 123k is running with process ID 9876, then running pv -R 9876 -L 321k will cause it to start using a rate limit of 321k instead of 123k. Note that some options cannot be changed while running, such as -c, -l, and -

Solaris 10: Adding a file system to a running zone

Since the global zone uses loopback mounts to present file systems to zones, adding a new file system was as easy as loopback mounting the file system into the zone’s file system:

$ mount -F lofs /filesystems/zone1oracle03 /zones/zone1/root/ora03

Once the file system was mounted, I added it to the zone configuration and then verified it was mounted:

$ mount | grep ora03/filesystems/zone1oracle03 on filesystems/zone1oracle0 read/write/setuid/devices/nonbmand/exec/xattr/atime/dev=2d9000b on Sun Apr 12 10:43:19 2009 /zones/zone1/root/ora03 on /filesystems/zone1oracle03 read/write/setuid/devices/dev=2d9000b on Sun Apr 12 10:44:07 2009

With ZFS filesystem (mountpoint=legacy):
mount -F zfs zpool/fs /path/to/zone/root/fs

Linux: remount read only file system

$ mount -o remount,rw /

Once you can write to the file system you should be able to write out changes to the file system to correct the issue that prevented the server from booting. Viva la remount!

Solaris: coreadm core file management

Using the Solaris coreadm utility to control core file generation

Solaris has shipped with the coreadm utiltiy for quite some time, and this nifty little utility allows you to control every facet of core file generation. This includes the ability to control where core files are written, the name of core files, which portions of the processes address space will be written to the core file, and my favorite option, whether or not to generate a syslog entry indicating that a core file was generated.

To begin using coreadm, you will first need to run it wit the “-g” option to specify where core files should be stored, and the pattern that should be used when creating the core file:

$ coreadm -g /var/core/core.%f.%p

Once a directory and file pattern are specified, you can optionally adjust which portions of the processes address space (e.g., text segment, heap, ISM, etc.) will be written to the core file. To ease debugging, I like to configure coreadm to dump everything with the”-G all” option:

$ coreadm -G all

Since core files are typically created at odd working hours, I also like to configure coreadm to log messages to syslog indicating that a core file was created. This can be done by using the coreadm “-e log” option:

$ coreadm -e log

After these settings are adjusted, the coreadm “-e global” option can be used to enable global core file generation, and the coreadm utility can be run without any arguments to view the settings (which are stored in /etc/coreadm.conf):

$ coreadm -e global

$ coreadm

global core file pattern: /var/core/core.%f.%p
     global core file content: all
       init core file pattern: core
       init core file content: default
            global core dumps: enabled
       per-process core dumps: enabled
      global setid core dumps: disabled
 per-process setid core dumps: disabled
     global core dump logging: enabled

Once global core file support is enabled, each time a process receives a deadly signal (e.g., SIGSEGV, SIGBUS, etc.):

$ kill -SIGSEGV 4652

A core file will be written to /var/core:

$ ls -al /var/core/*4652

-rw-------   1 root     root     4163953 Mar  9 11:51 /var/core/core.inetd.4652

And a message similar to the following will appear in the system log:

Mar 9 11:51:48 fubar genunix: [ID 603404 kern.notice] NOTICE: core_log: inetd[4652] core dumped: /var/core/core.inetd.4652

This is an amazingly useful feature, and can greatly simplify root causing software problems.

Solaris: killing defunct process

$ ps -ef | grep defunct

 root 646 426 0 - ? 0:00 ' 

 root 1489 12335 0 09:32:54 pts/1 0:00 grep defunct
$ preap 646
646: exited with status 0

This will cause the process to exit, and the kernel can then free up the resources that were allocated by that process.

Solaris: undelete file

How to undelete any open, deleted file on linux / solaris

Chris Dew wrote up a neat trick on how to recover files if deleted on Linux, yet still open by a process.

This works on Solaris as well.  =)

$:~:uname -a
SunOS somehost.com 5.10 Generic_127112-11 i86pc i386 i86pc

$:~:echo “sup prefetch.net folks?” > testfile
$:~:tail -f testfile &
[1] 17134

$:~:rm testfile
$:~:ls /proc/17134/fd/
0  1  2
$:~:cat /proc/17134/fd/0
sup prefetch.net folks?
$:~:cp !$ ./testfile
cp /proc/17134/fd/0 ./testfile
$:~:cat testfile
sup prefetch.net folks?

Solaris: using samba to access windows server folder

Accessing Windows shares from the Solaris/Linux command line

If Samba is installed on the system, this is easy to do with the smbclient utility. To access the Windows server named “milton” from the command line, you can run smbclient with the “-U” option, the name of the user to authenticate with, and the name of the server and share to access:

$ smbclient -U “domain\matty” //milton/foo

In this example, I am authenticating as the user matty in the domain “domain,” and accessing the share foo on the server milton. If smbclient is unable to resolve the server, you will need to make sure that you have defined a WINS server, or the server exists in the lmhosts file. To define a WINS server, you can add a line similar to the following (you can get the WINS server by looking at ipconfig /all on a Windows desktop, or by reviewing the LAN traffic with ethereal) to the smb.conf file:

wins server = 1.2.3.4

If you don’t want to use WINS to resolve names, you can add an entry similar to the following to the lmhosts file:

192.168.1.200 milton

Once you are connected to the server, you will be greeted with a “smb: \>” prompt. This prompt allows you to feed commands to the server, such as “pwd,” “dir,” “mget,” and “prompt.” To retrieve all of the files in the directory foo1, I can “cd” into the foo1 directory, use “prompt” to disable interactive prompts, and then run “mget” to retrieve all files in that directory:

smb: \> pwd
Current directory is \\server1\foo

smb: \> dir

received 10 entries (eos=1)
  .                                  DA        0  Mon May 22 07:19:21 2006
  ..                                 DA        0  Mon May 22 07:19:21 2006
  foo1                               DA        0  Sun Dec 11 04:51:12 2005
  foo2                               DA        0  Thu Nov  9 09:48:40 2006
         < ..... >

smb: \> cd foo1

smb: \foo1\> prompt
prompting is now off

smb: \foo1\> mget *

received 38 entries (eos=1)
getting file \foo1\yikes.tar of size 281768 as yikes.tar 411.3 kb/s) (average 411.3 kb/s)
    < ..... >

smb: \foo1\> exit

The smbclient manual page documents all of the available commands, and provides a great introduction to this super useful utility. If you bump into any issues connecting to a remote Windows server, you can add “-d” and a debug level (I like debug level 3) to the smbclient command line. This is perfect for debugging connectivity issues.