Timothy E. Archer

Recently I had to fix a linux system whose root file system was allowed to fill up. Not having much knowledge of the system, I needed a quick way to find out which files were consuming the most space to see if any of those could be purged. The likely culprit was some huge log files, but how to find them?

du and sort to the rescue!

This simple command dumped out the sizes of all files on the system:

du -x / | sort -rn | more

Let’s pick it apart:
continue reading…

Tonight I ran across the following piece written by Isaac Asimov, and felt is was worth sharing:

What is intelligence, anyway? When I was in the army, I received the kind of aptitude test that all soldiers took and, against a normal of 100, scored 160. No one at the base had ever seen a figure like that, and for two hours they made a big fuss over me. (It didn’t mean anything. The next day I was still a buck private with KP – kitchen police – as my highest duty.)
continue reading…

Recently, in an MBA class I was taking (Human Resources Management), I came across the 6 rules Jack Welch lives by.

For those who have no idea who Jack Welch is, he was Chairman and CEO of General Electric between 1981 and 2001. He practiced some unique leadership strategies during his tenure at GE, and increased its market capitalization by over $400 billion

I felt that the 6 rules are worth sharing with the the world and am publishing them below through this blog. By the way, I just finished the program and earned my MBA degree! Yay for me!

The six rules are:

1. Face reality as it is, not as it was or as you wish it to be.
2. Be candid with everyone.
3. Don’t manage, lead.
4. Change before you have to.
5. If you don’t have a competitive advantage, don’t compete.
6. Control your own destiny, or someone else will.

Recently I added a fraud alert to my credit report to see what exactly it does.

I went to http://www.experian.com/fraud and added an Initial Security Alert (90 days) to my credit report. Surprisingly, this was very easy to do, and only took me about 10 minutes. In any case, I got a free peek at who’s been viewing my credit, what accounts I have open, etc. I was able to verify that there are no other accounts in my name (thankfully, there were none), and I even submitted a dispute that a card that I am no longer an authorized purchaser on is still listed in my credit. Within 2 weeks that card was removed from my credit report. It is important to note that you will not be shown your FICO score when submitting a fraud alert. You can pay a small amount to have it displayed though.

After submitting the fraud alert, I also received a confirmation in the mail from Experian, TransUnion, and Equifax that they have put fraud alerts on my file which will last 90 days. It seems Experian notified the other big credit bureaus on my behalf. This was a nice bonus.
continue reading…

So you keep extending your Oracle datafiles, but do you know what objects are the true space hogs? I use the query below, to look at the size of the objects in a specific tablespace.

SELECT owner,
       segment_name,
       segment_type,
       tablespace_name,
       bytes
FROM DBA_SEGMENTS
WHERE TABLESPACE_NAME = 'DEVELOPMENT'
ORDER BY bytes desc

This will give me a list of my tables, indexes, and other objects in the specified tablespace in order by their size in bytes. The largest objects will be displayed first based on my sort criteria.

From this output, I usually see various tables that I can start purging, such as log tables, temp data tables, etc. This part will be specific to your own environment, but you’ll want to delete data you don’t need anymore.

Next, you need to compress the table extents and rebuild any indexes so that the space they used in the datafiles can be reused by other objects. It is not enough to just delete the data in the tables. You must either export/import the table, or use the much easier “alter table xxx move” syntax.

The two simple commands to do this are:

1. alter table tablename move;
2. alter index indexname rebuild;

So, let’s say you have a log table named activity_log that is 1 GB in size. You delete all but the last 3 months of log data from in, but the table is still consuming the 1 GB. To pack the table and compress its extents you would run the command:

alter table activity_log move;

Now, that table probably had a few indexes on it that are also consuming a large chuck of space. We’ll rebuild these to reclaim their space. It is important to note that your indexes will become invalidated after doing the alter table move, so you’ll have to rebuild your indexes.

alter index act_event_code_date rebuild;

Lastly, I’ll re-analyze the tables that I did mass deletes on and recompute the statistics for them:

analyze table activity_log compute statistics;

One important note is that the alter table move syntax does not work on tables with BLOB columns. You’ll have to do your standard export/import to compress these tables.

From the expect man page:

Expect is a program that “talks” to other interactive programs according to a script. Following the script, Expect knows what can be expected from a program and what the correct response should be. An interpreted language provides branching and high-level control structures to direct the dialogue. In addition, the user can take control and interact directly when desired, afterward returning control to the script.

At the institution for which I work, I am responsible for our Sungard HE Banner ERP system. One of the development tasks that I was presented with was to automatically run a binary file at 10pm every night. The challenge was that this program was interactive and required the user to input answers to prompts. This sounded like a perfect use of the Unix program expect.

If you are not a Sungard HE Banner ERP user, then skip the next two paragraphs. They’ll just be babble to you, and I’m sure you want to get to the good stuff about expect. For all of you Sungard HE Banner users reading this, the program I am calling is TGRCLOS. TGRCLOS is used to automatically close cashiering sessions, and I wanted to have the system close a certain set of cashiering sessions automatically at 10pm every night. This was so that the cashiering session we open for our online credit card payments that post directly into accounts receivable automatically closes at the same time our credit card processor (Intellipay) settles the transactions. Essentially, this was so that the cashiering session total matched what our credit card processor showed as being posted to a specific day.

By the way, someday I’ll post about how we developed our own poor man’s online credit card processing framework. This allows students to go into our portal, pull up their tuition bill, and pay online using a credit card. The payment also posts into Banner AR in real time. Its an awesome solution that really costs us nothing, just our standard credit card processing fees. It’s not the full blown TouchNet solution, but then again it didn’t cost us tens of thousands of dollars.

Now for the good stuff. Below is an overview of how I used expect. Keep in mind that this is all running on a RedHat AS 3 server, and if you don’t already have the expect binary installed then you will need to install it. My expect binary is in:

/usr/bin/expect

The first thing to do when you want to use expect to automatically interact with your program is to get all of the input prompts that the program asks for.

So I manually run the program I want to automate from my shell prompt. The command I want to automatically run is below, and I manually key that into my shell prompt to run it. Yours will probably be different.

tgrclos -o /home/banner/scripts/log/tgrclos.lis tarcher/mypassword

And the interactive output from my program looks like:

Connected.
Starting TGRCLOS (Release 7.3)

Run Sequence Number: 

(C)lose, (F)inalize Cashier Sessions or (B)oth: B

Run in (A)udit or (U)pdate Mode: U

Cashier Exception: 

Detail Category Code Exception: 

Close Sessions Containing Deposits [Y/N]: Y

Close Sessions Containing Misc. Transactions [Y/N]: Y

Number Of Printed Lines Per Page [55]: 

Enter "y" to proceed,
      "n" to respecify options,
      "x" to cancel report:      y

tgrclos completed successfully
87 lines written to /home/banner/scripts/log/tgrclos.lis

You can see that I had to answer a few prompts, and then the last two lines of output show the program running.

Now comes the fun part, writing the expect script to automatically interact with this program for me. Here are the steps:

1.First create a file with the extension .exp to hold your expect script.

vi run_tgrclos_for_sid.exp

2.Now in this file create the commands to spawn and interact with your program. My expect script looks like:

#!./expect -f
set timeout 30
spawn tgrclos -o /home/banner/scripts/log/tgrclos.lis $env(USERID)
expect "Run Sequence Number"
send "\r"

expect "(B)oth"
send "B\r"

expect "(U)pdate Mode"
send "U\r"

expect "Cashier Exception:"
send "\r"

expect "Detail Category Code Exception"
send "\r"

expect "Close Sessions Containing Deposits"
send "Y\r"

expect "Close Sessions Containing Misc. Transactions"
send "Y\r"

expect "Number Of Printed Lines"
send "\r"

expect "to cancel report:"
send "y\r"

#delay until a spawned process terminates
wait
The set timeout line just gives expect a timeout of 30 seconds in case it doesn’t find the prompt it is looking to fill in.

Then you need to spawn your program: This line in the expect script is what starts my program tgrclos working.

spawn tgrclos -o /home/banner/scripts/log/tgrclos.lis $env(USERID)

The -o /home/banner/scripts/log/tgrclos.lis $env(USERID) components are actually arguments to the tgrclos program I am running. It is nothing special to the expect script.

Also note that I use the special format $env(USERID) to get my userid and password string from an environment variable.

The next step is to write your expect script to expect a prompt, and then top provide a response. This is done through the series of expect and send lines.

For example, in my script I have:

expect "Run Sequence Number"
send "\r"

This tells expect to wait for the prompt text labeled Run Sequence Number, and then to just provide a carriage return (equivalent to just hitting the enter key) response.

Repeat your expect and send commands until you provided input to your entire program.

The last component of the expect script is the wait line which tells the expect program to wait until the spawned process terminates. We want to allow the TGRCLOS program to have a chance to finish running.

3.The last step is then to run your script. I do this through the commands:

export USERID=usfbanner/mypassword
expect run_tgrclos_for_sid.exp

You probably wont need the USERID environment variable export. I just needed it because of the $env(USERID) in my expect script.

Now you have a very simple overview of how to use expect. It can do A LOT more, and I’ve only scratched the surface of what can be done with this post.

Today I had a slight crisis…I had to reinstall QuickBooks 2006 on a new computer and restore the backup file that QuickBooks creates with the “.qbb” extension.

Everything worked great, and it was literally as simple as popping in the QuickBooks CD, installing the software, launching the program, and then following the steps from the “File->Restore menu”.

However, once I went into the Loan Manager to make a loan payment, I discovered none of the loans that I setup were there. I learned the hard way that the Loan Manager data is not backed up with your QuickBooks backup file. I really did not want to re-key in my loans, figure out what payment number I was on, determine principal and interest components, etc…Needless to say I was a little sad and looked for a way to get my Loan Manager data back.
continue reading…

I just got a new computer and had to figure out how to bring over all of my Firefox bookmarks and saved passwords over to it from my old computer.

For reference, my old and new computers are both running Windows XP, and my old computer is running Mozilla Firefox 2.0.0.3.

In order to bring over all of my personal settings to my new computer I performed the following actions:

1. First install Firefox on the new computer.
2. On your old computer, navigate to the directory containing all of your personal settings. We want to copy this directory from your old computer to your new one.
   This folder is located in your Documents and Settings folder for the user that you are logged into your computer as. On my computer this folder is named:

   C:\Documents and Settings\tarcher\Application Data\Mozilla\Firefox\Profiles\nkf6289t.default

   So, I grabbed the folder named nkf6289t.default and all files and directories within it, and copied it to a temporary directory on my new computer. Make sure you have FireFox closed on your computer before you start the copy.

3. On my new computer the profile is stored in the directory named:

   C:\Documents and Settings\tarcher\Application Data\Mozilla\Firefox\Profiles\o75144ji.default

   This is what Firefox created for me as when I installed it.

   On my new computer, I simply moved the existing directory C:\Documents and Settings\tarcher\Application Data\Mozilla\Firefox\Profiles\o75144ji.default out of the Profiles directory, and moved the directory in that I copied from my old computer.

4. Make sure you rename the directory you are copying in the same as what it was named on your new computer. So, everything I copied over went into the folder named C:\Documents and Settings\tarcher\Application Data\Mozilla\Firefox\Profiles\o75144ji.default
5. Startup Firefox on your new computer. All of your bookmarks and saved passwords should be back in place and functioning.