allow tftp under selinux

For RHEL5:

# audit2allow -a -M mytftp
# semodule -i mytftp.pp
# service xinetd restart

add virtual hard disk to linux

#in vmware workstation | vm settings
add virtual hard disk to vm

#in vm
fdisk /dev/sdb
n
[ENTER][ENTER]
w
mkfs -t ext3 /dev/sdb1
mkdir /newdir
mount -t ext3 /dev/sdb1 /newdir
echo "/dev/sdb1 /software ext3 defaults 1 1" >> /etc/fstab

See: http://www.matttopper.com/?p=25

remount /

mount -n -o remount /

tw_cli: 3ware controller commands

when replacing a failed disk, you must issue a 'maint deleteunit command', as the 'maint remove' command doesnt do what you might think it does. so, pull the drive, insert the new one, issue a rescan commmand, then deleteunit, then rebuild.

the following commands were used to start rebuiding the raid-1 array on a server with a single 8006 controller.

show ver
info
info c0
rescan
info c0
maint deleteunit c0 u1
maint rebuild c0 u0 p1

list pecl modules and info

You might have to:
`ln -s /usr/local/apache2/bin/apxs /usr/local/bin/apxs`

Then:
`/usr/local/apache2/php/bin/pecl list`
`/usr/local/apache2/php/bin/pecl install apc`
`vi /usr/local/apache2/php/conf/php.ini`
- add 'extension=apc.so'
`/usr/local/apache2/bin/apachectl_admin restart`

list kernel modules

# modinfo $(cut -d' ' -f1 /proc/modules) | sed '/^dep/s/$/\n/; /^file\|^desc\|^dep/!d'

host firewall for the mac

#!/bin/sh
#fwrules

IPFW='/sbin/ipfw -q'

$IPFW -f flush
$IPFW add 2000 allow ip from any to any via lo*
$IPFW add 2010 deny log ip from 127.0.0.0/8 to any in
$IPFW add 2020 deny log ip from any to 127.0.0.0/8 in
$IPFW add 2030 deny log ip from 224.0.0.0/3 to any in
$IPFW add 2040 deny log tcp from any to 224.0.0.0/3 in
$IPFW add 2050 allow log tcp from any to any out
$IPFW add 2060 allow tcp from any to any established
$IPFW add 12190 deny log tcp from any to any


Then apply it to the firewall

# sudo ipfw list
# chmod 600 ./rules
# sudo ipfw ./rules

pecl install apc

make sure that the apxs directory is in the system path.

resize a mounted lvm-managed disk in linux

I have found many overly-complex and incorrect and/or unnecessary instructions all over the web. So, I made this reference for me, but it may work well for you too. I typically use RHEL4/5 in my data center implementations, so these steps cover lvm managed disk space (which is decent for a simple LAMP stack).

I tend to do the following a lot in vmware products (workstation/esx/esxi).

First, check and note the size of your disks and partitions in your target vm.

[root@host]# df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      5.8G  665M  4.9G  12% /
/dev/sda1              99M  9.1M   85M  10% /boot
none                  506M     0  506M   0% /dev/shm

[root@host]# sfdisk -s
/dev/sda:   8385898
/dev/sda1:    104391
/dev/sda2:   8281507
/dev/dm-0:   6160384
/dev/dm-1:   2031616

Then, go into the settings of your vm (e.g. through the vi client) and either add another virtual hard disk or increase the size of the existing disk. In my case i simply extended the existing virtual disk from 8G to 16G.

Reboot.

Check to see that the size of /dev/sda has increased.

[root@host]# sfdisk -s
/dev/sda:  16777216
/dev/sda1:    104391
/dev/sda2:   8281507
/dev/dm-0:   6160384
/dev/dm-1:   2031616

[root@host]# sfdisk -l

Disk /dev/sda: 2088 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+     12      13-    104391   83  Linux
/dev/sda2         13    1043    1031    8281507+  8e  Linux LVM
/dev/sda3          0       -       0          0    0  Empty
/dev/sda4          0       -       0          0    0  Empty

/* If you increased the size of your existing disk, its probably easiest to boot from the gparted-live disk and create a new primary partition in the newly free space, format ext3. */
wget http://downloads.sourceforge.net/project/gparted/gparted-live-stable/0.4.6-1/gparted-live-0.4.6-1.iso?use_mirror=softlayer

reboot

/* check for /dev/sda3, your new parition */

[root@host]# sfdisk -s
/dev/sda:  16777216
/dev/sda1:    104391
/dev/sda2:   8281507
/dev/sda3:   8385930
/dev/dm-0:   6160384
/dev/dm-1:   2031616

/* create the lvm pv reference */

[root@host]# pvcreate /dev/sda3

/* extend the lvm vg reference */

[root@host]# vgextend VolGroup00 /dev/sda3

/* note the free space for your upcoming `lvextend` command */

[root@host]# vgdisplay

  --- Volume group ---
  VG Name               VolGroup00
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  4
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               15.84 GB
  PE Size               32.00 MB
  Total PE              507
  Alloc PE / Size       250 / 7.81 GB
  Free  PE / Size       257 / 8.03 GB
  VG UUID               L7woQB-ymCv-NeWL-i47M-b5Ua-fOHQ-hM0DXI

/* extend the lvm vg reference */

[root@host]# lvextend -L+8.03G /dev/VolGroup00/LogVol00

/* resize the volume group while its still mounted */

[root@host]# ext2online /dev/VolGroup00/LogVol00

check the following

[root@host]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       14G  668M   13G   6% /
/dev/sda1              99M  9.1M   85M  10% /boot
none                  506M     0  506M   0% /dev/shm

[root@host]# sfdisk -l

Disk /dev/sda: 2088 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+     12      13-    104391   83  Linux
/dev/sda2         13    1043    1031    8281507+  8e  Linux LVM
/dev/sda3       1044    2087    1044    8385930   83  Linux
/dev/sda4          0       -       0          0    0  Empty

now, go on with your life.

compare directory trees

ssh server1 "find /usr/local/apache2/sites/htdocs/ -type f -exec basename {} \; | sort" > server1.txt; ssh server2 "find /usr/local/apache2/sites/htdocs/ -type f -exec basename {} \; | sort" > server2; comm -3 ./server1 ./server2

slowloris ddos aversion

use Nginx and openBSD/pf to protect Apache.
http://nginx.net/

here's some configuration help.
https://calomel.org/nginx.html

rpm packages by name only

rpm -qa --qf "%{NAME}\n" > hostname.rpm.txt

then you can compare to see what is missing.

comm -3 host1.rpm.txt host2.rpm.txt

Slowloris DDOS prevention

#!/bin/sh

LIMIT=100

COMMAND='netstat -n | egrep '\''tcp.*[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*:(80|443)[ ]*[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*:[0-9]*[ ]*ESTABLISHED
'\'' | awk -F'\''[ \t:]+'\'' '\''{ print $6 }'\'''

eval $COMMAND | sort | uniq -c | while read numconn ip
do
if [ $numconn -gt $LIMIT ]
then
echo "Check ASAP and renable this cron." | mail -s "IP $ip - ($numconn) went over $LIMIT connections on `hostname`" me@someemailaddress.com

sed -i 's/\(^\*.*this_script.sh*\)/#\1/g' /etc/crontab
# /sbin/iptables -I INPUT -s $ip -j DROP
fi
done

ESXi 4.0 not booting off DVD ISO or Host CD/DVD Device with DVD's

I had some problems with booting off DVD iso's on nfs shares and DVD's in the Host system CD/DVD reader (which, incidentally is a SATA device, also problematic with ESXi according to some forums I've read). But I was able to get it to boot off the DVD media by burning the DVD iso to a physical disk and placing the disk in my client system DVD reader. Then, I set the boot options to delay by 5000ms, set the boot order in the VM's bios as removable,network,cdrom,hard drive, and then i started the vm. while it was sitting there trying to boot off the network, i used the little CD button above the vm console window to connect my local DVD drive to the vm through the VI4 client. This worked, it was slow but it worked and I was so happy!

List installed perl modules

perl -MCPAN -e 'print CPAN::Shell->r '

convert unix time to local time

date -R -d @1245049200

-R requests that date output in RFC 2822 format
-d requests that date output the date based on a string that follows

Set up SNMP v3 on Cisco IOS

conf t
snmp-server group group_name v3 priv
snmp-server group group_name v3 priv read secure_ro write secure_rw access 5
snmp-server view secure_ro internet included
snmp-server view secure_rw mgmt included
snmp-server user snmp_user iksecure v3 auth md5 auth_password priv des56 priv_password

access-list 5 permit host x.x.x.x
access-list 5 deny any log


show snmp group
show snmp user

Create a CA and a Signed Cert

To make certificate authority:

mkdir CA
cd CA
mkdir certs crl newcerts private
echo "01" > serial
cp /dev/null index.txt
cp /usr/local/openssl/openssl.cnf.sample openssl.cnf
vi openssl.cnf   (set values)
openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 -config openssl.cnf

To make a new certificate:

cd CA        (same directory created above)
openssl req -nodes -new -x509 -keyout newreq.pem -out newreq.pem -days 365 -config openssl.cnf

(certificate and private key in file newreq.pem) To sign new certificate with certificate authority:

cd CA        (same directory created above)
openssl x509 -x509toreq -in newreq.pem -signkey newreq.pem -out tmp.pem
openssl ca -config openssl.cnf -policy policy_anything -out newcert.pem -infiles tmp.pem
rm -f tmp.pem

(newcert.pem contains signed certificate, newreq.pem still contains unsigned certificate and private key)

Installing sshfs on RHEL5

installing sshfs on rhel5
yum install kernel-devel gcc
wget http://downloads.sourceforge.net/fuse/fuse-2.7.4.tar.gz
tar xvf fuse-2.7.4.tar.gz
cd fuse-2.7.4
./configure
make
make install
modprobe fuse
echo "modprobe fuse" > /etc/sysconfig/modules/fuse.modules
ls -la
wget http://dag.wieers.com/rpm/packages/fuse-sshfs/fuse-sshfs-1.9-1.el5.rf.i386.rpm
wget http://dag.wieers.com/rpm/packages/fuse/fuse-2.7.3-1.el5.rf.i386.rpm
rpm -Uvh ./fuse-2.7.3-1.el5.rf.i386.rpm
rpm -Uvh ./fuse-sshfs-1.9-1.el5.rf.i386.rpm

Remove nameserver references from resolv.conf

sed -i 'N;$!P;$!D;$d' /etc/resolv.conf

Howto Shrink a VM

First, run a script similar to the following, one for each significant mountpoint as defined in /etc/fstab

#!/bin/sh

cd /
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /tmp
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /home
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /var
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill
cd /usr
cat /dev/zero > zero.fill;sync;sleep 1;sync;rm -f zero.fill

# Delete this script

rm -rf /usr/local/jboss/server/default/log/*
cd /root/tools
rm -rf shrink.sh

Then shutdown the vm and close vmware workstation.
Run the following command at the Windows Command Interpreter.

vmware-vdiskmanager -k <path to vmdk>

chroot tor in openBSD

https://wiki.torproject.org/noreply/TheOnionRouter/OpenbsdChrootedTor

config check

#!/bin/sh
#written and tested on openbsd 4.4
#pf.master contains the last known good sha1 of pf.conf

MASTER=`cat ./pf.master`
DGST=`ssh fwmon@192.168.43.132 'sudo sha1 /etc/pf.conf | cut -d" " -f 4'`

#echo $DGST
#echo $MASTER

if [[ "${MASTER}X" == "${DGST}X" ]] ; then

#notify that unscheduled config has changed
#insert incident details into security db
echo "fw Config Secure"

else

#insert incident details into security db
echo "fw Config Breached"

fi

sed && awk one-liners

These guides are great. Thanks to all who compiled them.

Famous sed one-liners Explained, Part I
Famous sed one-liners Explained, Part II
Famous sed one-liners Explained, Part III

Famous awk one-liners Explained, Part I
Famous awk one-liners Explained, Part II
Famous awk one-liners Explained, Part III

Convert Windows CR/LF to Unix newlines

sed 's/.$//'

sed 's/^M$//'

Delete lines from a file by line number with sed

Today i decided to make a new monitoring tool, and I needed to make a list of all permutations of 3 in a set of 22. The set happens to be hostnames of a private Tor network. Order is important, as forming 3-hop circuits through Tor is sequential, which is why i need permutations instead of combination's.

22 * 21 * 20 = 9240 permutations

Crap, I'm not really up on my combinatorial number theory, I guess I'll have to hack it up.

First I used an excel plugin to generate all the permutations.

But this ended up giving me 1408 invalid permutations, because the mix of sets had 10647 results. I copied the results into a text file and counted the number of lines as well as obtained the line numbers of the invalid permutations using this script:
----------
#!/bin/sh

tornames=("tornode01" "tornode02" "tornode03" "tornode04" "tornode05" \
"tornode06" "tornode07" "tornode08" "tornode09" "tornode10" \
"tornode11" "tornode12" "tornode13" "tornode14" "tornode15" \
"tornode16" "tornode17" "tornode18" "tornode19" "tornode20" \
"tornode21" "tornode22")

for i in ${tornames[*]};
do
while read line; do echo $line|tr " " "\n"|grep $i |wc -l; done < ./vc_list.bak > ./lines.$i
grep -rn '3\|2' ./lines.$i | cut -d: -f1 > ./lines.$i.ln
done
----------

It saved a bunch of files for me as: lines.[hostname], containing a number on each line indicating the number of times the hostname appears on each line.
Then it grep'd out the lines with a 2 or a 3, asking grep to return the line number, and cut the line number from the output to a file named: lines.[hostname].ln

Then at the command line I did this:

# cat ./lines.*.ln > line.numbers.all
# sed 's/.*/&d/g' ./line.numbers.all > ./delete.sed
# sed -f delete.sed ./file.master >> file.trimmed

Using a sed delete file...finally I had my 9240 valid permutations:

# cat ./file.trimmed | wc
9240 27720 254520

Next I want to make this text list into an array that I can `source` into the monitoring script as an array.

# rsync ./file.trimmed ./perms_array.sh
sed -i -e 's/^\./perms_array.sh

Almost done, I just need to fill in the array number with another sed expression.

# sed = ./perms_array.sh | sed 'N; s/^// ; s/\nperms\[// ; s/^/perms\[/' > \
./perms_array.final.sh

and now to put quotes around the array value:

# sed -e 's/\=/\=\"/' < ./perms_array.final > ./perms_array.final.new && rsync ./perms_array.final.tmp ./perms_array.final
# sed -e 's/$/\"/' < ./perms_array.final > ./perms_array.final.new && rsync ./perms_array.final.tmp ./perms_array.final

here's what the file looks like:

perms[1]="tornode01 tornode02 tornode03"
perms[2]="tornode01 tornode04 tornode05"
perms[3]="tornode01 tornode06 tornode07"
...

Now I can move on to write an essentially simple script that performs the test of all possible virtual circuits.

oops

Today I was trying to move a file into the home directory of the current user, like this:

# mv /home/otheruser/somefile ~

Interestingly enough, after doing this for the last 15 years, I fat-fingered it like this:

# mv /home/otheruser/somefile !

CRAP!
Guess what, my file was deleted.
Btw, Mac OSX doesnt behave like this. I assume *BSD, Solaris and other high-quality systems dont as well. Pooor Linux.

SNMPv3 Quickstart

i DONT claim this to be complete or authoritative. But, with these quick steps i was able to get snmpv3 working, a generally avoided version of a widely used protocol, and a fog to many sysadmins i've worked with. I'm sick of reading 10 pages of prose to get the steps i need to move securely forward in my projects.

0) yum install net-snmp.i386 net-snmp-libs.i386

1) Run snmpconf -i to create snmpd.conf and snmp.conf
- sudo /usr/bin/snmpconf -i

Notes:
- if you're regenerating the files at some point, cd into /etc/snmp, then run `snmpconf -i`. snmpconf looks in the local dir for files first before looking elsewhere. The resultant files are still saved to /usr/local/share/snmp/
- when creating snmp.conf, complete section 3: 2-10
- when creating snmpd.conf, complete section 4: 1-3
- when creating a new user while configuring snmpd.conf, choose 'priv' for the minimum security level. you can also restrict the user to a specific branch of the OID tree here as well.

2) Copy these files to /etc/snmp

rsync -av /usr/local/share/snmp/snmp.conf /etc/snmp/
rsync -av /usr/local/share/snmp/snmpd.conf /etc/snmp/

3) Run net-snmp-config to actually create snmpv3 user, here is the correct syntax...

net-snmp-config --create-snmpv3-user [-ro] [-A authpass] [-X privpass] [-a MD5|SHA] [-x DES|AES] [username]

Here's my command that corresponds to my previous configuration of snmp.conf and the test snmpget command further below in step 5:

net-snmp-config --create-snmpv3-user -ro -A authpass -X privpass -a SHA -x AES rouser

Note: the manpage for net-snmp-config has the X and x incorrectly in their example of create-snmpv3-user. The help cruft (for net-snmp-config --help) shows it correctly. I tried to create a read-write user (with -rw), but it didnt work. I dont change system parameters through snmp anyways, so it doesnt matter to me. Maybe the absence of [-ro] creates a read-write user? seems like ro should be the default unless -rw is specified. ???

4) Restart snmpd service

5) Make test snmpv3 request

snmpget -v 3 -n "" -u rouser -a SHA -A "authpass" -x AES -X "privpass" -l authPriv localhost system.sysUpTime.0

Notes:
- The `-l authPriv` argument specifies that the request should be both signed (-a SHA) and encrypted (-x AES).
- The command above can be greatly simplfied because most of these options have been declared in the snmp.conf.

install a perl module

perl -MCPAN -e 'install Net::SNMP'

or

perl -MCPAN -w -e 'shell'

CPAN> install Net::SNMP

Deployment Tools: Puppet

I've started looking at Puppet as the next gen tool for system deployments. Check it out.

http://reductivelabs.com/trac/puppet/wiki/DocumentationStart

It leaves cfengine and others in the dust...

List all Perl Modules

perl -MFile::Find=find -MFile::Spec::Functions -Tlwe "find { wanted => sub { print canonpath $_ if /\.pm\z/ }, no_chdir => 1 }, @INC"

Passmark Health Check

curl https://localhost:443/pmws_server/healthCheck -k

Running VMware ACE Player as a Windows Service

First, read this. I wasnt able to use the resource kit tools because Macrosh@ft wont allow you to redistribute their tools. So, we instead bought a tool that offered an OEM license. If you dont know what ACE is, its vmware's option pack for VMware Workstation. Its basically a bunch of security features and packaging options, meaning, you can build vm and then package it up and install it on another system. I wanted to use it because I wanted to make sure the server could not be copied and run somewhere else by anyone except the people I choose and authorize.

One of the features in an ACE policy is that allows you to run a script or executable instead of enter a password when the vm is started. The VMware ACE Player, interestingly enough, will accept a string from STDOUT of this script or exe in order to attempt the decryption of the encrytion key thats used to read the vmdk files as the vm runs. Using FireDaemon and the FireDaemon features of Pre-Post Service commands, I was able to use vmrun.exe stop command to shut down the vm when the host system is rebooted. I also made lanmanworkstation a service that the Firedaemon service depends on. This way, the vm is not started until the network is full up on the host system...other people have mentioned using this technique and it seemed like a good one.

The executeable, written in VC++ STL, accepts two args, a meaningful 9 digit number and a secret. If either of these args fail a number of tests performed on them, the exe quietly exits. If the args pass, a SHA512 hash (using openssl) is performed and a 64 character string is printed out to STDOUT. Its this string that the vmplayer.exe uses to encrypt/decrypt the AES keys that encrypt/decrypt the vmdk's when the vm starts.

These files, the exe and the openssl libs (dll's) and the VC manifest and their dll's and any other script and stuff you want to use in the Firedaemon service configuration, reside in the "ACE Resources" directory, under the parent directory of the vm Master.

Also, i got snagged by this: each time a package is built, the packager drops an ace.sig file into the "ACE Resources" directory of the package it generates. Make sure this ace.sig file doesnt get copied back into the Master "ACE Resources" directory. If it does, all the packages that you make will have this invalid file in there. Its easy to get into this situation when you delete the files in that directory in order to make a test package that may be updated with a policy update package, which would contain the authentication module and scripts. VMware should fix this by checking for this file and deleting it from within the Master's dir structure each time the packager runs, but it doesn't at this time. Anyhow, there is a policy option called Resource Signing that does check this file, and if its set to check it, which it is by default, the activation of the package (or policy update) will fail.

There was one more terribly annoying thing with setting up the service. When I attempted to shutdown or reboot the host system (unforced), the vmplayer.exe would abort the shutdown/reboot operation and display a modal dialog box that says "virtual machine is in use." I got around this by using AutoIT, a freeware application that allows you to create a simple script to operate windows and applications. So far its been a breeze and a very sensible and intuitive scripting language. Good docs help a lot too. Anyhow, this script ran as a Firedaemon pre-service and sits there waiting for that stupid dialog box to activate. It checks for this window every 250ms. Then, when the shutdown sequence starts, it gets rid of the dialog box and reboots the host system. Pretty slick.

Its too bad vmware says they wont support ACE vm's anytime soon on ESX or VMware Server. It would be pretty simple to get this working and I think that vm volumes that remain encrypted on-disk at all times solves a very difficult security challenge in operating system virtualization. no longer can someone make a copy of your entire filesystem and mount it somewhere else.

How to view pflog

Viewing the pflog file:
# tcpdump -n -e -ttt -r /var/log/pflog

A real-time display of logged packets:
# tcpdump -n -e -ttt -i pflog0

 

Install a package manager in MacOSX (Darwin Ports) and install GnuPG port

First, get the package manager:

$ wget http://www.portcode.com/darwinports/DarwinPorts-1.5.0-10.4.dmg
or
$ curl http://www.portcode.com/darwinports/DarwinPorts-1.5.0-10.4.dmg -O

Mount dmg image and install, then open a Terminal window:

$ sudo port -d selfupdate
$ cd /opt/local/var/macports/

$ port search gnupg
$ sudo port install gnupg
Password:
$ gpg --gen-key

conf file is: ~/.gnupg*, add keyservers, change your default fingerprint, and other stuff there.

Deep Water Swell for Northern California

http://cdip.ucsd.edu/?nav=recent⊂=nowcast&units=metric&tz=UTC&pub=public&xitem=sf

Geologic Overlays of Northern California for Google Earth

http://geomaps.wr.usgs.gov/sfgeo/geologic/downloads.html

Squid Fact Sheet

http://www.tonmo.com/science/public/giantsquidfacts.php