There must be something useful here…

Recently we have been tasked to provide SVN support with Bluehost.  Initially I was reading and found it was not possible to complete.  After a bit more searching I found that it was possible.  The steps here are not as simple as “apt-get install svn”, we actually have to compile it from source.  Since I have been extremely busy lately, I tasked my friend Henry for the job and then told him I could provide support when needed.

With the tutorial in hand, he went to work.  Here is the steps outlined in the tutorial:

cd ~
mkdir src
cd ~/src
wget http://www.gtlib.gatech.edu/pub/apache/apr/apr-util-1.2.12.tar.gz
wget http://www.gtlib.gatech.edu/pub/apache/apr/apr-1.2.12.tar.gz
wget http://subversion.tigris.org/downloads/subversion-1.5.4.tar.gz
wget http://www.webdav.org/neon/neon-0.28.0.tar.gz
tar -xzf apr-util-1.2.12.tar.gz
tar -xzf apr-1.2.12.tar.gz
tar -xzf subversion-1.5.4.tar.gz
tar -xzf neon-0.28.0.tar.gz
cd ~/src/apr-1.2.12
./configure --prefix=$HOME LDFLAGS="-L/lib64"
make
make install
cd ~/src/apr-util-1.2.12
./configure --prefix=$HOME --with-apr=$HOME LDFLAGS="-L/lib64"
make
make install
cd ~/src/neon-0.28.0
./configure --enable-shared --prefix=$HOME LDFLAGS="-L/lib64"
make
make install
cd ~/src/subversion-1.5.4
./configure --prefix=$HOME --without-berkeley-db  --with-ssl LDFLAGS="-L/lib64"
make
make install

After making it through all of the above configuring and compiling, the binaries are sitting in ~/system/bin ready for use.

Now to add a new repo, type:

svnadmin create /[dir]/[repo name]

Now import files to the repo:

svn import /[dir]/[repo name] [svn string, starting with svn://]

Now the repo is ready to add files or to check them out. But, you will soon learn that you need to add somethings to the path in order to properly execute. Once I added the things to the path, it still didn’t work. I found out that bluehost doesn’t allow execution of the .bashrc if it is a single execution item. I found a work around:

First you will have to add a public key to the .ssh/authorized_keys from your system so that you can login without using a password.  If you need help with that, here is an awesome tutorial that works very well! SSH Key Tutorial

Add the following before your rsa key in ~/.ssh/authorized_keys

command="/home/USERNAME/system/bin/svnserve -t" ssh-rsa [KEY REMOVED]

Now you should be able to login using the CLI for the system that you have added the key for and use SVN.

Now that you have everything else setup, it is time to get TurtoiseSVN client up and running.

First install Putty, if you need help with that, click here.

For those that didn’t click the link, click it for fun .

Next you are going to need to get WinSCP or Filezilla or some other program that can do file transfer over SSH.  I prefer Filezilla because it is Filezilla.  If you don’t have either of those, simply Google them and you’ll find them.

Ok, now you will need to log in to your Bluehost server with Putty.  Once you have logged in, type the following:

ssh-keygen -t rsa

This will create 2 files id_rsa and id_rsa.pub in the .ssh/ directory. First you will want to copy the public key to make it so you can use this key on this server, type the following:

cp .ssh/id_rsa.pub .ssh/authorized_keys

Now I will assume you are using Filezilla. Open Filezilla and then for the hostname, use “sftp://[domainname.com]” and your username and password. Navigate to the ~/.ssh directory and download id_rsa to a directory on your computer of your choosing.

When you downloaded putty, you should have also downloaded PuttyGen. Open PuttyGen and then select the Conversions menu and select Import Key. Find the id_rsa file you just downloaded and import it. After it is imported select the “Save Private Key” button and export the .PPK file to your choosing.

Now we are going to open Putty and save a new session with our newly created key. Once Putty is opened, go to SSH and then to AUTH and then select the private key you exported above. After that go back to the sessions option and type the whole domainname (including .com/.net/.whatever) as the “Saved Session” and push the Save button. It is important to save the session because Tortoise will use this in just a bit.

You can test your SSH login to make sure it doesn’t ask for a password and you will know that you are on the right path.

Next up, we have to make some changes to your Tortoise config file located at C:\Documents and Settings\YOUR_LOCAL_USERNAME\Application Data\Subversion\config
If you have Vista or Win7, it will be located here:
C:\Users\YOUR_LOCAL_USERNAME\AppData\Roaming\Subversion\config
Now find the section called [tunnels] and add the following:

ssh = c:\\Program\ Files\\TortoiseSVN\\bin\\TortoisePlink.exe

Link in Linux, you have to escape the backslashes with a backslash.

Ok, you should be ready to rock at this point. Time to install Tortoise if you haven’t already. Download it here. Follow the installer, should be easy.

Time to check something out. Try to checkout a repo now by right clicking in Windows Explorer and choosing “SVN Checkout.” Use the following string but add your own info:

    svn+ssh://youraccount@[domain]/home/[your account]/[repo path]/[Project]

Hopefully everything worked, if it didn’t, hit me up and I’ll help out.

Mike

Over the last few weeks I have had an all out war with a Cisco ASA.  Typically ASA’s are not that hard to configure.  Some of the ACL’s can get tricky, but it is quite easy to determine if they are working or not.  Now when you are trying to figure out how to make a VPN work and the other side has assured you that their configuration and configuration instructions are fine, it clouds the issues.  To begin, I will give you some background on the configuration.  This particular device is guarding the entry point for a network that has Remote-Access users and is also required for Site-to-Site to another companies network.  The remote access configuration was previously configured and working.  It was my responsibility to figure out the issue with the site-to-site and make the VPN work.
Initially, it seemed simple, take traffic over the VPN, but then NAT it before it gets to the other side.  Easy right?  No No, this turned into a total mess.  The issue was that the instruction read that “you need to NAT interesting traffic to 10.1.0.205/32 (255.255.255.255).”  They also gave us the information that, “Your Local Network: 10.1.0.205/32 (255.255.255.255),” which is supposed to be our NAT side of the VPN.  Ok, so let me break this down…  You want me to take my network traffic (192.168.1.0/24) and then NAT it to 10.1.0.205/32 on the 10.1.0.205/32 network???  This is the assumption that was throwing me off course.  Their instructions should have been better to read, “Your Local Network: 10.1.0.205/24 (255.255.255.0)” which means that we have a host on a network, not a host on a host.

So anyways, here is the special modifications that got thing working for me:

access-list outside_cryptomap_1 extended permit ip host 10.1.0.205 192.168.50.0 255.255.255.0
access-list outside_cryptomap_1 extended permit ip 192.168.1.0 255.255.255.0 192.168.50.0 255.255.255.0
global (inside) 10 10.1.0.205
nat (inside) 10 access-list outside_cryptomap_1
crypto map outside_map 1 match address outside_cryptomap_1

These rules provided the meat to connect the site-to-site VPN.

In the process to the above solution I learned a number of very valuable things that will save me on future ASA configurations, like for example, when creating a Remote-Access VPN from the CLI, you must have the dynamic crypto map rules,  you must have crypto isakmp enabled, and you must define a crypto isakmp policy.  If you do not have any of these things the VPN will not negotiate.  If you are going to use DHCP, then you will need to have a dhcpd pool and it is also vital to have a group-policy.

Well, The war is over and I have won!  Many hours have been spent on this project, but I believe I can conquer nearly anything on the Cisco ASA!

-Mike

Being a programmer by nature (and by degree) and I have dabbled with a lot of different computer languages. Some I fluently code in, others are nice for rapid development, and some are for someone else. I have been working on moving some Perl web applications to a new server. Typically this wouldn’t be too big of a deal. You make sure you have all the modules, make sure you have the correct permissions, make sure you have Apache configured to do CGI, and then you move the files and everything should work. In this instance, someone forgot to read ‘Modular Programming and Best Practices for the Perl language,’ a book surely someone has written. The code I am working on does not make very much use of the CGI-BIN directory, it has Perl scattered all through out HTML files, and there are not any comments or direction. The worst part is that the programmer chose to use Microsoft Visual C++ variable names to name Perl variables. For example, a string in this script is $m_lpszData… This is wrong on so many levels. The only thing I am going to point out is that Perl doesn’t explicitly use pointers, that’s all I’m going to say. If this last piece doesn’t make sense to you, then you have had a very sheltered programming career.

Now the main point of adding this entry is to have somethings in here that have been major breakthroughs for me. The first is understanding that Internal Error 500 doesn’t mean anything at all. In fact, it has to be the worst error message ever created. Who would create a serious error message and then not give any detail as to why the error happened. I understand on a production server, but offer some options to spew debugging madness.

After much research it is apparent that the Internal Error doesn’t receive any data that would help with debugging. Which is a sort of safe guard for the webserver. As soon as the fatal error happens, it nukes the script and then spew that information. Most of the time, the information in the logs (/var/log/apache/user/default.log if using virtualhosts or ISPCP) is not very helpful and quite vague.

Despite the lack of information I figured I would use an old C trick to print out each line and variable to see where the issue was. The problem is that with Internal Error 500, you don’t get any return at all. So I did one better, I was able to use Carp, a debugging framework for Perl. That with randomly place ‘die’ statements I was able to follow the flow of code execution. The issue is that I wasn’t able to track down the real reason for the software bug. Finally I stumbled upon CGI::Debug. This module rules. It figured out a way to supersede the Apache Internal error and it gave me all the information about the running script and exactly where it died. With this information I was able to see that I was never printing the header information out, which was never creating a proper webpage for Apache to serve up. I added a few print statements and was able to get the expected functionality.

Now as I continue on my journey of Perl debugging, I found a few other tips that are very helpful, use perl -wc [script name] to compile the script and check it. Use perl -d [script name] to start the Perl debugger which allows you to step through code. Lastly, you can use rm * -f if you get stuck… Just kidding, don’t do that unless you have a backup copy or know how to recover deleted files. Hopefully I am able to wrap this project up, I am ready to be done with Perl. In my opinion, PHP is much more flexible, intuitive, and powerful.

Mike

I used to have a Wiki that I would keep track of all of the “Cool Commands” that I found or created.  But now that Wiki is down and I haven’t had the time to put it back up.  So, I need to keep track of commands so I don’t forget them.   Alright, starting off, have you ever wanted to change a few lines of text in a bunch of files?  I have.  I found a one line Perl script that will take care of it for you, and leave you a backup file for your trouble.

• perl -pi -i.bak -e ‘s/searchval/replaceval/’ *.html

This command can use regular expressions for the search value and the replace value.

This next command will sync your information and can be used to only keep updated copies of files.   There are lots of options too, this only scratches the surface.

• rsync -uav --progress /srcdir/ /dstdir/

This command will will look at the srcdir and then determine if the dstdir file is newer, if it is, it won’t copy.  Very useful if you need to merge multiple copies of files, but don’t want duplicates.

This next command will find files for you based on all sorts of criteria:

• find . -mtime -10

This command will look for files that are -10 days old.

You can combine rsync and find to search for files and then sync those files.

• find . -mtime -10 -print0 | rsync -av --progress --files-from=- -from0

This command will first compile a list of files that meet the find criteria and then will port them to rsync.  Very helpful.

Ok, that is all the cool commands for today, I will probably find some more tomorrow.

Found more… This command will make a file of any size for you:

• dd if=/dev/zero of=file.out bs=1MB count=100

This command will create a 100 meg dummy file. Works great.

Mike

For the issues I have been having with the above formula, I decided to write up my experience in hopes that I can either help someone or at least provide a few little details to make some sense of things.

First off, I just want to say, that if you, as a programmer, choose to use Perl, and then further choose to design a full web application with Perl, PLEASE, PLEASE use the CGI-BIN file so that for future programmers on your project may be able to move your project without an act of congress.  KEEP THINGS SIMPLE!!!

Ok, here is the preface of the situation, A company I have been working with for a few months has a large number of antiquated servers.  They purchased some new ones and then wanted to move everything to the new servers.  This seems like a not too difficult task, the source machines are Fedora.  I opted for the newest release of Ubuntu because of the many new features like integrated cloud control and the fact that it is built on Debian.  Don’t boo me since I am taking the easy road, there is a Sonic wall that will do the filtering and provide a very high level of security and IDS.  Since this is not a post on security I won’t get into the details of my security plan.

Once LAMP and IspCP was all setup, it was time for the copy.  After I had a complete copy of all the files, I began filling out the holes of the new, empty webserver.  For anyone who has messed with IspCP, you will know that it uses /var/www/virtual/[domain name]/htdocs to store the web accessible files.  The cgi-bin folder is stored at the same level as htdocs.

I began populating htdocs with the info from the previous server and everything starting going fine.  Even all of the PHP files worked and executed properly thanks to settings in IspCP.  The issue, Perl was sprinkled throughout much of the web applications.  Many database calls and functions were all written in Perl.  One would think that this isn’t a big deal and that with a few apt-get calls, everything would work out.

The major issue is that it was the intention of the “Elders of The Internet” to store executable files in the cgi-bin.  And from my research (which is many hours at this point), it seems Apache2 with Virtual Hosts is dedicated to following these directives.  After hours of messing with the apache2.conf, ispcp.con, and other config files, I finally have found a work around that is not convenient, but workable.

Inside the virtual host file ispcp.conf (in /etc/apache2/site-enabled/ispcp.conf) under the <VirtualHost> directive, you can place:

AddHandler cgi-script .pl .html
Options +ExecCGI

Inside the same file, inside the <Directory *> directive you can place the same information as above to have it apply only to the specific directory.  However, this only works with directories at the same level or deeper in the tree. (Eg. /here or /here/nowhere).  When I would try to use a directory outside the VirtualHost document root, I would get an Internal Error 500 with the suexec.log file saying that the directory had other write permissions enabled…. Very strange.

The things that don’t work, and guarantee Internal Error 500 are adding the <file ~ .pl> with the file handlers for perl scripts to apache2.conf.  If you try to add an ‘*’ to the directory in the above example, it causes a 500 error.

Finally, I found out the hard way that make changes to the ispcp.conf file don’t stay if someone adds a new website or addon.  Instead you must modify the following file to make it persistent.  /etc/ispcp/apache/working/ispcp.conf

Now back to adding the top directive to the subdirectories.

***UPDATE:  It seems that the previous practice I prescribed earlier has failed.  When I added all of the info to the ispcp.conf, it gave me another Internal Error 500.  This time it was saying a file was writeable by others.  To compound the issue, it looks like Firefox and IE were both reporting Cached pages.  I found a setting in IE to only pull fresh, new copies everytime.  I figured a CTRL+F5 would give me a fresh copy.  It looks like the solution was to add AddHandler cgi-script .pl .html Options +ExecCGI to the .htaccess file.  But now when I add it to a subdomain I am back to the stupid issue with the internal error 500.  I am going to bed.

Good Luck,

Mike