host firewall for the mac

#!/bin/sh
#fwrules

IPFW='/sbin/ipfw -q'

$IPFW -f flush
$IPFW add 2000 allow ip from any to any via lo*
$IPFW add 2010 deny log ip from 127.0.0.0/8 to any in
$IPFW add 2020 deny log ip from any to 127.0.0.0/8 in
$IPFW add 2030 deny log ip from 224.0.0.0/3 to any in
$IPFW add 2040 deny log tcp from any to 224.0.0.0/3 in
$IPFW add 2050 allow log tcp from any to any out
$IPFW add 2060 allow tcp from any to any established
$IPFW add 12190 deny log tcp from any to any


Then apply it to the firewall

# sudo ipfw list
# chmod 600 ./rules
# sudo ipfw ./rules

pecl install apc

make sure that the apxs directory is in the system path.

resize a mounted lvm-managed disk in linux

I have found many overly-complex and incorrect and/or unnecessary instructions all over the web. So, I made this reference for me, but it may work well for you too. I typically use RHEL4/5 in my data center implementations, so these steps cover lvm managed disk space (which is decent for a simple LAMP stack).

I tend to do the following a lot in vmware products (workstation/esx/esxi).

First, check and note the size of your disks and partitions in your target vm.

[root@host]# df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      5.8G  665M  4.9G  12% /
/dev/sda1              99M  9.1M   85M  10% /boot
none                  506M     0  506M   0% /dev/shm

[root@host]# sfdisk -s
/dev/sda:   8385898
/dev/sda1:    104391
/dev/sda2:   8281507
/dev/dm-0:   6160384
/dev/dm-1:   2031616

Then, go into the settings of your vm (e.g. through the vi client) and either add another virtual hard disk or increase the size of the existing disk. In my case i simply extended the existing virtual disk from 8G to 16G.

Reboot.

Check to see that the size of /dev/sda has increased.

[root@host]# sfdisk -s
/dev/sda:  16777216
/dev/sda1:    104391
/dev/sda2:   8281507
/dev/dm-0:   6160384
/dev/dm-1:   2031616

[root@host]# sfdisk -l

Disk /dev/sda: 2088 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+     12      13-    104391   83  Linux
/dev/sda2         13    1043    1031    8281507+  8e  Linux LVM
/dev/sda3          0       -       0          0    0  Empty
/dev/sda4          0       -       0          0    0  Empty

/* If you increased the size of your existing disk, its probably easiest to boot from the gparted-live disk and create a new primary partition in the newly free space, format ext3. */
wget http://downloads.sourceforge.net/project/gparted/gparted-live-stable/0.4.6-1/gparted-live-0.4.6-1.iso?use_mirror=softlayer

reboot

/* check for /dev/sda3, your new parition */

[root@host]# sfdisk -s
/dev/sda:  16777216
/dev/sda1:    104391
/dev/sda2:   8281507
/dev/sda3:   8385930
/dev/dm-0:   6160384
/dev/dm-1:   2031616

/* create the lvm pv reference */

[root@host]# pvcreate /dev/sda3

/* extend the lvm vg reference */

[root@host]# vgextend VolGroup00 /dev/sda3

/* note the free space for your upcoming `lvextend` command */

[root@host]# vgdisplay

  --- Volume group ---
  VG Name               VolGroup00
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               15.84 GB
  PE Size               32.00 MB
  Total PE              507
  Alloc PE / Size       250 / 7.81 GB
  Free  PE / Size       257 / 8.03 GB
  VG UUID               L7woQB-ymCv-NeWL-i47M-b5Ua-fOHQ-hM0DXI

/* extend the lvm vg reference */

[root@host]# lvextend -L+8.03G /dev/VolGroup00/LogVol00

/* resize the volume group while its still mounted */

[root@host]# ext2online /dev/VolGroup00/LogVol00

check the following

[root@host]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       14G  668M   13G   6% /
/dev/sda1              99M  9.1M   85M  10% /boot
none                  506M     0  506M   0% /dev/shm

[root@host]# sfdisk -l

Disk /dev/sda: 2088 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+     12      13-    104391   83  Linux
/dev/sda2         13    1043    1031    8281507+  8e  Linux LVM
/dev/sda3       1044    2087    1044    8385930   83  Linux
/dev/sda4          0       -       0          0    0  Empty

now, go on with your life.

compare directory trees

ssh server1 "find /usr/local/apache2/sites/htdocs/ -type f -exec basename {} \; | sort" > server1.txt; ssh server2 "find /usr/local/apache2/sites/htdocs/ -type f -exec basename {} \; | sort" > server2; comm -3 ./server1 ./server2

slowloris ddos aversion

use Nginx and openBSD/pf to protect Apache.
http://nginx.net/

here's some configuration help.
https://calomel.org/nginx.html

rpm packages by name only

rpm -qa --qf "%{NAME}\n" > hostname.rpm.txt

then you can compare to see what is missing.

comm -3 host1.rpm.txt host2.rpm.txt

Slowloris DDOS prevention

#!/bin/sh

LIMIT=100

COMMAND='netstat -n | egrep '\''tcp.*[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*:(80|443)[ ]*[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*:[0-9]*[ ]*ESTABLISHED
'\'' | awk -F'\''[ \t:]+'\'' '\''{ print $6 }'\'''

eval $COMMAND | sort | uniq -c | while read numconn ip
do
if [ $numconn -gt $LIMIT ]
then
echo "Check ASAP and renable this cron." | mail -s "IP $ip - ($numconn) went over $LIMIT connections on `hostname`" me@someemailaddress.com

sed -i 's/\(^\*.*this_script.sh*\)/#\1/g' /etc/crontab
# /sbin/iptables -I INPUT -s $ip -j DROP
fi
done

ESXi 4.0 not booting off DVD ISO or Host CD/DVD Device with DVD's

I had some problems with booting off DVD iso's on nfs shares and DVD's in the Host system CD/DVD reader (which, incidentally is a SATA device, also problematic with ESXi according to some forums I've read). But I was able to get it to boot off the DVD media by burning the DVD iso to a physical disk and placing the disk in my client system DVD reader. Then, I set the boot options to delay by 5000ms, set the boot order in the VM's bios as removable,network,cdrom,hard drive, and then i started the vm. while it was sitting there trying to boot off the network, i used the little CD button above the vm console window to connect my local DVD drive to the vm through the VI4 client. This worked, it was slow but it worked and I was so happy!

List installed perl modules

perl -MCPAN -e 'print CPAN::Shell->r '

convert unix time to local time

date -R -d @1245049200

-R requests that date output in RFC 2822 format
-d requests that date output the date based on a string that follows

Set up SNMP v3 on Cisco IOS

conf t
snmp-server group group_name v3 priv
snmp-server group group_name v3 priv read secure_ro write secure_rw access 5
snmp-server view secure_ro internet included
snmp-server view secure_rw mgmt included
snmp-server user snmp_user iksecure v3 auth md5 auth_password priv des56 priv_password

access-list 5 permit host x.x.x.x
access-list 5 deny any log


show snmp group
show snmp user

Create a CA and a Signed Cert

To make certificate authority:

mkdir CA
cd CA
mkdir certs crl newcerts private
echo "01" > serial
cp /dev/null index.txt
cp /usr/local/openssl/openssl.cnf.sample openssl.cnf
vi openssl.cnf   (set values)
openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 -config openssl.cnf

To make a new certificate:

cd CA        (same directory created above)
openssl req -nodes -new -x509 -keyout newreq.pem -out newreq.pem -days 365 -config openssl.cnf

(certificate and private key in file newreq.pem) To sign new certificate with certificate authority:

cd CA        (same directory created above)
openssl x509 -x509toreq -in newreq.pem -signkey newreq.pem -out tmp.pem
openssl ca -config openssl.cnf -policy policy_anything -out newcert.pem -infiles tmp.pem
rm -f tmp.pem

(newcert.pem contains signed certificate, newreq.pem still contains unsigned certificate and private key)

Installing sshfs on RHEL5

installing sshfs on rhel5
yum install kernel-devel gcc
wget http://downloads.sourceforge.net/fuse/fuse-2.7.4.tar.gz
tar xvf fuse-2.7.4.tar.gz
cd fuse-2.7.4
./configure
make
make install
modprobe fuse
echo "modprobe fuse" > /etc/sysconfig/modules/fuse.modules
ls -la
wget http://dag.wieers.com/rpm/packages/fuse-sshfs/fuse-sshfs-1.9-1.el5.rf.i386.rpm
wget http://dag.wieers.com/rpm/packages/fuse/fuse-2.7.3-1.el5.rf.i386.rpm
rpm -Uvh ./fuse-2.7.3-1.el5.rf.i386.rpm
rpm -Uvh ./fuse-sshfs-1.9-1.el5.rf.i386.rpm

Remove nameserver references from resolv.conf

sed -i 'N;$!P;$!D;$d' /etc/resolv.conf

Howto Shrink a VM

First, run a script similar to the following, one for each significant mountpoint as defined in /etc/fstab

#!/bin/sh

cd /
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /tmp
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /home
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /var
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /usr
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill

# Delete this script

rm -rf /usr/local/jboss/server/default/log/*
cd /root/tools
rm -rf shrink.sh

Then shutdown the vm and close vmware workstation.
Run the following command at the Windows Command Interpreter.

vmware-vdiskmanager -k <path to vmdk>

chroot tor in openBSD

https://wiki.torproject.org/noreply/TheOnionRouter/OpenbsdChrootedTor

config check

#!/bin/sh
#written and tested on openbsd 4.4
#pf.master contains the last known good sha1 of pf.conf

MASTER=`cat ./pf.master`
DGST=`ssh fwmon@192.168.43.132 'sudo sha1 /etc/pf.conf | cut -d" " -f 4'`

#echo $DGST
#echo $MASTER

if [[ "${MASTER}X" == "${DGST}X" ]] ; then

#notify that unscheduled config has changed
#insert incident details into security db
echo "fw Config Secure"

else

#insert incident details into security db
echo "fw Config Breached"

fi

sed && awk one-liners

These guides are great. Thanks to all who compiled them.

Famous sed one-liners Explained, Part I
Famous sed one-liners Explained, Part II
Famous sed one-liners Explained, Part III

Famous awk one-liners Explained, Part I
Famous awk one-liners Explained, Part II
Famous awk one-liners Explained, Part III

Convert Windows CR/LF to Unix newlines

sed 's/.$//'

sed 's/^M$//'

Delete lines from a file by line number with sed

Today i decided to make a new monitoring tool, and I needed to make a list of all permutations of 3 in a set of 22. The set happens to be hostnames of a private Tor network. Order is important, as forming 3-hop circuits through Tor is sequential, which is why i need permutations instead of combination's.

22 * 21 * 20 = 9240 permutations

Crap, I'm not really up on my combinatorial number theory, I guess I'll have to hack it up.

First I used an excel plugin to generate all the permutations.

But this ended up giving me 1408 invalid permutations, because the mix of sets had 10647 results. I copied the results into a text file and counted the number of lines as well as obtained the line numbers of the invalid permutations using this script:
----------
#!/bin/sh

tornames=("tornode01" "tornode02" "tornode03" "tornode04" "tornode05" \
"tornode06" "tornode07" "tornode08" "tornode09" "tornode10" \
"tornode11" "tornode12" "tornode13" "tornode14" "tornode15" \
"tornode16" "tornode17" "tornode18" "tornode19" "tornode20" \
"tornode21" "tornode22")

for i in ${tornames[*]};
do
while read line; do echo $line|tr " " "\n"|grep $i |wc -l; done < ./vc_list.bak > ./lines.$i
grep -rn '3\|2' ./lines.$i | cut -d: -f1 > ./lines.$i.ln
done
----------

It saved a bunch of files for me as: lines.[hostname], containing a number on each line indicating the number of times the hostname appears on each line.
Then it grep'd out the lines with a 2 or a 3, asking grep to return the line number, and cut the line number from the output to a file named: lines.[hostname].ln

Then at the command line I did this:

# cat ./lines.*.ln > line.numbers.all
# sed 's/.*/&d/g' ./line.numbers.all > ./delete.sed
# sed -f delete.sed ./file.master >> file.trimmed

Using a sed delete file...finally I had my 9240 valid permutations:

# cat ./file.trimmed | wc
9240 27720 254520

Next I want to make this text list into an array that I can `source` into the monitoring script as an array.

# rsync ./file.trimmed ./perms_array.sh
sed -i -e 's/^\./perms_array.sh

Almost done, I just need to fill in the array number with another sed expression.

# sed = ./perms_array.sh | sed 'N; s/^// ; s/\nperms\[// ; s/^/perms\[/' > \
./perms_array.final.sh

and now to put quotes around the array value:

# sed -e 's/\=/\=\"/' < ./perms_array.final > ./perms_array.final.new && rsync ./perms_array.final.tmp ./perms_array.final
# sed -e 's/$/\"/' < ./perms_array.final > ./perms_array.final.new && rsync ./perms_array.final.tmp ./perms_array.final

here's what the file looks like:

perms[1]="tornode01 tornode02 tornode03"
perms[2]="tornode01 tornode04 tornode05"
perms[3]="tornode01 tornode06 tornode07"
...

Now I can move on to write an essentially simple script that performs the test of all possible virtual circuits.