All Entries Tagged With: "kung"

Advanced Process Whack-a-Mole

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Ed Prepares to Open Up a Can of Process Whoop-Ass:

I’ve never considered myself a particularly violent man. But, I have to admit it: Sometimes it just feels good to kill processes. I’ve even been heard to mutter a deadpan “Dodge This” in my lab late at night as I obliterate errant or evil processes just begging to meet their maker. Then, to make sure such a process doesn’t pop back up to start bothering me again, I sow the ground with command-line kung fu salt to strangle any other similar process that might pop up in its place.

This technique, which we’ve taken to calling “Process Whack-a-Mole”, can be helpful to people in all walks of life. I’m sure it’s happened to pretty much everyone at some point. You find yourself playing defense in a Capture the Flag tournament against an elite team of ninjas from a three-letter government agency who want to completely control your boxen. They repeatedly gain access, and you have to shew them out before they score and you lose points. To deal with such situations, we can run a command to continuously look for processes with certain characteristics of our adversaries, and then kill them when they appear. We touched upon the idea of killing a process that starts to listen on a given TCP port in Episode #76. But, let’s go further this time, discussing how you can make much more flexible whack-a-mole commands to deal with various process characteristics.

These techniques are useful even outside of Capture the Flag games. I often use them in malware analysis and even system administration when I want to suppress some activity temporarily while I’m analyzing or configuring something else.

The basic structure I use for process whack-a-mole consists of the following three parts:

<Continuous Loopinator> <Process Selector> <Process Terminator>

Quite often the Process Selector and Process Terminator are combined together in a single command, because we can select or filter for the process we want in the same command we use to whack it. However, to filter for certain specific process characteristics, we’ll have to split out these two entities. I’ll show you what I mean in a bit.

We start out with our Continuous Loopinator:

C:\> for /L %i in (1,0,2) do @ping -n 2 127.0.0.1 >nul

This is a simple FOR /L loop that starts counting at 1, goes up to 2, in steps of 0. In other words, it’s the cmd.exe equivalent of while (true), used to keep something running continuously. At the start of the loop, we introduce a 1-second delay by pinging ourselves twice (-n 2) and throwing the standard output away so as not to clutter our output (>nul). That way, we’ll run our Process Selector and Process Terminator approximately every 1 second, helping to minimize our impact on performance. If you want a faster Process Selector, simply omit that ping, and your system will run our whack-a-mole command as fast as it can, but performance may drag.

We then follow with our Process Selector. If you keep in the ping delay, put in an & followed by the Process Selector, which lets us make one command run after another. Otherwise, just put the Process Selector after the @ (which turns off command echo, by the way… no sense having our output clogged up with commands).

The two most common Process Selectors I use are wmic and taskkill, which have the nice property of also including the ability to act as Process Terminators in the same command. Let’s look at wmic first.

The wmic command can be used to select given processes based on our constructing a where clause, using the following syntax:

C:\> wmic process <where clause> <verb clause>

In the where clause, we can specify any attribute or group of attributes of processes that can be listed via wmic process. To get a list of these attributes, you could run:

C:\> wmic process get /?

So, for example, if you want to select a processID of 4242, you could write your wmic command as:

C:\> wmic process where processid=4242

Or, we could look for processes that have a given Parent Process ID:

C:\> wmic process where parentprocessid=3788

Or, we could look for processes with a given name:

C:\> wmic process where name="cmd.exe"

These where clauses also support AND and OR, but you’ve got to make sure you put the guts of your where clause inside of parentheses. The where clauses also support not equals (!=). Check this out:

C:\> wmic process where (name="cmd.exe" and processid != 676)

Now, we haven’t supplied a verb clause here, so all our wmic commands are simply displaying raw, unformatted process information on Standard Output.

Let’s start doing our whack-a-mole by specifying a Process Terminator by using the verb clause of wmic with the simple verb “delete”. That’ll kill a process.

Putting these pieces together, suppose you want to kill all cmd.exe processes other than the current cmd.exe you, the administrator, are running. Let’s assume that your own cmd.exe has a process ID of 676. You could run:

C:\> for /L %i in (1,0,2) do @ping -n 2 127.0.0.1 >nul & wmic process where (name= "cmd.exe" and processid!=676) delete

Now, let’s see ‘em try to run a cmd.exe. Every time someone tries to launch one, your loop will kill it.

Next, suppose you want to prevent a given process with processid 4008 from spawning child processes. Maybe process ID 4008 is a cmd.exe shell, and you want to prevent the person who is using it from being able to run any commands that aren’t built-into the shell itself. Or, better yet, maybe process ID 4008 is Tim Medin’s PowerShell process, and you wanted to pee in his Corn Flakes, depriving him of the ability to run any separate EXE commands, forcing him to rely solely on built-in capabilities of Powershell itself. We can do this one without our ping-induced delay to really confound him:

C:\> for /L %i in (1,0,2) do @wmic process where parentprocessid=4008 delete

These wmic where clauses also support substring matches, with the use of “like” and %. For example, suppose you want to continuously kill every process that is running from an EXE with a path in a given user’s directory. You could run:

c:\> for /L %i in (1,0,2) do @ping -n 2 127.0.0.1 >nul & wmic process where (executablepath like "c:\\users\\tim\\%") delete

Note that in a “where” clause with the “like” syntax, you have to surround the elements with parens. Also, note that if you have a \ in your where clause, you have to specify it as \\, the first \ indicating an escape, and the second indicating your backslash.

You can combine these where clause elements (=, !=, AND, OR, LIKE, and %) in all kinds of ways to mix and match against various process attributes for whack-a-mole. I’m sure our readers can dream up all kinds of interesting and useful combinations.

But, there is a missing attribute from “wmic process get /?” — it’s the user name that process is running under. To play whack-a-mole based on user names, we can turn to another Process Selector and Terminator: taskkill. I wrote about taskkill filters back in Episode 22, showing how we can use it to kill a process based on its owner username. Here, we’ll wrap that in our whack-a-mole construct:

C:\> for /L %i in (1,0,2) do @ping -n 2 127.0.0.1 >nul & taskkill /F /FI "username eq tim"

Sorry, Tim. That’s what you get for using a shell on the same system I’m on. Hal’s over on some lonely Linux that no one ever uses, so I leave him alone. :)

Anyway, where was I? Ah, yes, we were discussing attributes of processes that wmic doesn’t include, but which may be handy in whack-a-mole games. How about DLLs? Suppose a bad guy is hacking your system and keeps trying to inject a DLL into some process, and you want to kill that process. Maybe Evil Badguy (yes, that’s his full name) has injected metsrv.dll, the Metasploit Meterpreter, into a running process, and uses process migration to jump from process to process. Sorry, but getting the system to unload that DLL using only built-in tools at the command line is very difficult, but killing that process is totally doable:

C:\> for /L %i in (1,0,2) do @ping -n 2 127.0.0.1 >nul & taskkill /F /FI "modules eq metsrv.dll"

Now, I mentioned above that the Process Selector and Process Terminator components are typically combined in a single command, as we’ve seen so far with wmic and tasklist. When would you have two different commands for these pieces? Well, one example is with netstat, which can show TCP and UDP port usage and the processes associated with each port. That’s exactly what we used in Episode #76, where our Process Selector was netstat (whose output I parsed with a FOR /F loop to pull out the ProcessID number), and the Process Terminator was taskkill:

C:\> for /L %i in (1,0,2) do @(ping -n 2 127.0.0.1>nul & for /f "tokens=5" %j in ('netstat -nao ^| find ^":47145^"') do @taskkill /f /PID %j)

So, keeping in mind those three components of process whack-a-mole, you can use almost any command that lists processes in pretty much any arbitrary way to build a whack-a-mole command for fun and profit.

And now, for a routine disclaimer: Be careful with any of these commands. If you kill a vital system process, such as lsass.exe, you could bring your whole box down. You have been warned. So, now that you are armed and dangerous, go have fun!

Tim prepares for war:

It seems that Ed has a bit of shell envy. So let’s kick that inferior shell off “our” machine and keep it (and him) off our machine.

As Ed described, the structure for whack-an-Ed whack-a-mole has three parts, and that basic structure will be very similar in PowerShell.

<Continuous Loopinator> { <Process Selector> | <Process Terminator> }

The Continuous Loopinator repeatedly calls the Process Selector whose results are piped into the Process Terminator. Let’s see how each piece works.

Continuous Loopinator:

There are many ways to do a continuous loop, but the easiest and quickest method is to use the While loop.

PS C:\> while (1) { <code to run ad infinitum>; Start-Sleep 1 }

This loop is pretty self explanatory. It is a simple While loop that runs while 1 is true, which it always will be. The Start-Sleep cmdlet (alias sleep) will suspend activity for the specified amount of time. If we wanted a shorter nap we could use the -milliseconds parameter. Since Ed’s command runs every second, we should run ours a bit faster just because we can. How about 5 times a second?

PS C:\> while (1) { <code to run ad infinitum>; Start-Sleep -milliseconds 200 }

Process Terminator:

I’m covering this a bit out of order because the Terminator is so simple, so indulge me for a bit. The cmdlet used for killing is Stop-Process (alias spps or kill). It can even be used for some rudimentary process selection before the assassination. We can kill based on the Process Id:

PS C:\> Stop-Process 1337

…or the process name.

PS C:\> Stop-Process -Name cmd

In the second example every process with the name “cmd” would be stopped, but what if we wanted to be a little more high tech in making Ed’s processes “sleep with the fishes?”

As described earlier, the results of the Process Selector can be piped into our Process Terminator. We can pick any method to retrieve the process(es) to be killed, but more on that later. Here is what it would look like:

<Get Processes> | Stop-Process

By default, Stop-Process will ask for confirmation prior to terminating any process not owned by the current user. To get around that safety mechanism we can use the Force parameter.

<Get Processes> | Stop-Process -Force

Short version:

<Get Processes> | kill -f

We could just kill the processes with Stop-Process by giving it a Process Id or process name, but we want more options. Now let’s see how we can find more processes to kill.

Process Selector:

To get a process or a number of processes we use Get-Process (aliases ps and gps). This is our Process Selector. We have covered this before, but we have a number of ways to get a process or a list of processes. To get help on the command you can run:

PS C:\> Get-Help Get-Process

…or for those who have seen the light and come from the linux side but have a bad memory, this works too:

PS C:\> man ps

To see the examples use the Examples parameter, or use the Full parameter to see everything. From the help we can see how to get a process with a given Process ID. We will be looking for PID 4242:

PS C:\> Get-Process -Id 4242 PS C:\> ps -Id 4242

To get all the cmd.exe processes:

PS C:\> Get-Process cmd PS C:\> ps cmd

Note that the process name does NOT include .exe.

We can also use filters in order to get more granular. We already had our loop to kill all cmd processes, but what if Ed wants to use PowerShell? We need to make sure that we are King of the PowerShell Hill, and any other PowerShell usurper is destroyed. This will find any PowerShell processes that aren’t ours.

PS C:\> Get-Process powershell | ? { $_.ID -ne 21876 }

The weaponized version of the command could look like this:

PS C:\> While (1) { ps powershell | ? { $_.ID -ne 21876 } | kill -f; sleep 1 }

The next thing Ed did, after tee-tee’ing in my Kelloggs, was to prevent me from kicking off any processes from a cmd or PowerShell process. So let’s do the same to him. Unfortunately, the objects returned by Get-Process do not have a Parent Process Id property, so we will have to use WMI to find processes with a given parent.

PS C:\> Get-WmiObject win32_process -Filter "ParentProcessId=5552" | % { Get-Process -Id $_.ProcessID }

Get-WmiObject (alias gwmi) is used to access WMI in order to get all processes with a Parent Process Id of 5552. The results are piped into a ForEach-Object (alias %) loop. In the loop we use Get-Process and the Process Id retrieved from WMI in order to get the process object. We can then pipe that into our kill(er). Similar to what Ed did, we want to run this continuously so he doesn’t have a chance. Here is what our command looks like:

PS C:\> While (1) { gwmi win32_process -Filter "ParentProcessId=5552" | % { ps -Id $_.ProcessID } | kill -f }

We also want to make sure that Ed isn’t able to run anything from his user directory (which includes his desktop).

PS C:\> While (1) { ps | ? { $_.Path -like "c:\users\ed\*" } | kill -f }

We use the Where-Object (alias ?) to filter our list of processes based on the path. The Like operator is used with our wildcard search string in order to find any of Ed’s processes. Again, we pipe the results into Stop-Process in order to kill it.

Just to make sure that Ed doesn’t run anything, we will kill any process where he is the owner. Again, we will have to use WMI in order to find the owner of a process.

PS C:\> While (1) { Get-WmiObject Win32_Process | ? { $_.GetOwner().User -eq "ed" } | % { Get-Process -Id $_.ProcessId } | Stop-Process -Force }

This command is a little complicated, so let’s break it down piece by piece. The While loop portion should be obvious so we’ll skip that bit of explanation. The first chunk…

Get-WmiObject Win32_Process | ? { $_.GetOwner().User -eq "ed" }

We start off by querying WMI and retrieving WMI objects representing each process running on the current machine. The results are then piped into our filter, Where-Object (alias ?). The “current pipeline object”, represented by the variable $_, allows us to access the properties of each object passed down the pipeline. For all intents and purposes, the $_ variable is used to iterate through each object passed down the pipeline. It takes the first object, in our case the first process, and accesses the GetOwner method’s User property. We then check to see if the value is equal (-eq) to “ed”. If it is equal, then our WMI object passes our filter and is sent further down the pipeline. Remember, the objects are WMI Process Objects, not PowerShell Process Objects, and they will need to be converted to the PowerShell version so we can natively deal with them in PowerShell. On to the conversion.

... | % { Get-Process -Id $_.ProcessId } ...

The objects that passed through the filter are now sent into our ForEach-Object (alias %) loop. This loop is used to iterate through each object and execute some fu on each of the WMI objects. Again, $_ represents the current object. To retrieve the PowerShell version of the process object we use Get-Process. We need to use Id parameter with the Process Id property of the current object ($_.ProcessId). Now we have PowerShell Process Objects. YAY!

... | Stop-Process -Force

Finally, the processes are piped into Stop-Process to be destroyed. The Force option is used since we don’t want a confirmation to kill each process.

Next, let’s look for the processes with the injected Meterpreter dll. How do we find this dll? We need to look at the modules a process has loaded. Here is what the Modules property looks like for the PowerShell process.

PS C:\> Get-Process powershell | select modules

Modules : {System.Diagnostics.ProcessModule (powershell.exe), System.Diagnostic s.ProcessModule (ntdll.dll), System.Diagnostics.ProcessModule (kernel 32.dll), System.Diagnostics.ProcessModule (KERNELBASE.dll)...}

As you can see the dll name is wrapped in parenthesis. So here is how we kill it.

PS C:\> Get-Process | ? { $_.Modules -like "*(metsrv.dll)*" } | Stop-Process

Actually, the modules property is a collection of module objects. So we can use a nested Where-Object to filter.

PS C:\> ps | ? { $_.Modules | ? {$_.ModuleName -eq "metsrv.dll" }} | kill

In this command we retrieve all the processes. We then filter the Modules, where the ModuleName is metsrv.dll. The results are piped into Stop-Process.

We can also parse netstat in order to kill a process similar to Episode #76. Let’s take that command and wrap it in our infinite loop.

PS C:\> While (1) { netstat -ano | ? {$_ -like "*:47145 *"} | % { $_ -match "\d+$"; stop-process $matches[0] } }

And as Ed said, be careful not to kill the wrong process or the whole box could go down. Of course, when it is down it is pretty dang hard to attack. Of course, it is also pretty dang hard to use too.

Now that Ed and I have spent all of our energy going after each other, Hal is going to show up and mop the floor with our tired carcases.

Disclaimer: No Eds where harmed in the making of this episode.

Hal’s Analysis:

Why are Ed and Tim so angry all the time? It couldn’t have anything to do with the platform they’ve chosen to work on, could it? Hey guys, don’t worry, be happy! You can always install Linux for free, or even just use Cygwin.

When Ed first proposed this topic, I was pretty stoked because I thought it was going to be a cake-walk for me with my little friend lsof. But not all of Ed’s challenges that could be answered purely with lsof. Some required a bit more shell fu.

Let’s start with the simple stuff first. The “infinite loop with 1 second delay” idiom for bash is something we’ve seen before in previous Episodes:

# while :; do [...your commands here...]; sleep 1; done

In this case, the commands we put into the while loop are going to be a kill command and usually some variant of “`lsof -t …`” we’ll be using to select the PIDs we want to kill. Remember from previous Episodes that “lsof -t” causes lsof to print out just the PIDs of the matching processes, specifically so we can use the output as arguments to the kill command.

For example, let’s suppose we want to kill all of Ed’s processes. We can use lsof’s “-u” option to select processes for a particular user:

# while :; do kill -9 `lsof -t -u skodo`; sleep 1; done

Or we could nuke all the bash shells on the machine, using “-c” to select commands by name:

# while :; do kill -9 `lsof -t -c bash`; sleep 1; done

Of course, this would hammer our own shell, so it pays to be more selective:

# while :; do kill -9 `lsof -t -a -c bash -u^root -u^hal`; sleep 1; done

Here I’ve added the “-a” flag which means do a logical “and” on my selection criteria. Those criteria are “all commands named bash” (“-c bash”) and “not user root” (“-u^root”) and “not user hal” (“-u^hal”). Note that lsof’s negation operator (“^”) only works when selecting user names, PIDs (with “-p”), process group IDs (with “-g”), command names (with “-c”), and protocol state info (“-s”, as in “-s^TCP:LISTEN”).

Another one of Ed’s challenges was killing processes where the binary is in a particular directory. Again we can do this with lsof:

# while :; do kill -9 `lsof -t -a -d txt +d /home/skodo`; sleep 1; done

Here we’re looking for process binaries using “-d txt”. In the lingo, the binary is what’s used to create the “text segment” of a process (where the executable code lives), hence “-d txt” for lsof. The “+d” tells lsof to look for open files under a particular directory. Yes, lsof has so many command line options that the author had to start doubling up on letters using “+” instead of “-” (there’s a reason the lsof manual page is nearly 50 pages long when printed out).

Note that “+d” only searches “one level deep”. So if Ed were running “/home/skodo/evil”, then our loop above would whack that process. But if Ed were running “/home/skodo/bin/evil”, then we wouldn’t catch it. If you want to do full directory recursion, use “+D” instead of “+d”. lsof distinguishes these with separate options because full directory searches are so time-consuming.

However, as I mentioned earlier, Ed had challenges that I wasn’t able to come up with a “pure lsof” solution for. For example, while lsof has the “-R” option for displaying parent process ID (PPID) values, there aren’t any switches in lsof to select particular processes by PPID. So we’ll need to resort to some awk:

# while :; do kill -9 `lsof -R -d cwd | awk '($3 == 8552) { print $2 }'`; sleep 1; done

Here the lsof command is outputting PPID values (“-R”) in addition to the normal lsof output, and we’re only outputting the lines showing the current working directory of each process (“-d cwd”). The “-d cwd” hack is a good way of ensuring that you only get one line of lsof output per process– so we don’t end up outputting the same PID multiple times and generating spurious error messages from kill. The awk code simply matches against a particular PPID value in column #3 and outputs the PID value in column #2.

Even though I had to resort to a bit of awk in the last example, you have to admit that having lsof makes this challenge unfairly easy for us Unix/Linux folks. Ahhh, lsof! How I love thee! Let me count the ways…

admin | Jan 19, 2010 | Comments 0

Yule Be Wanting an Explanation Then

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket
Hal returns to the scene of the crime

I opened last week’s post saying there would be no “explanations or excuses”, but apparently that wasn’t good enough for some of you. So at the request of our loyal readers, we’d like to revisit last week’s episode and explain some of the code. Especially that crazy cmd.exe stuff that Ed was throwing around.

Of course the bash code is completely straightforward:

$ ct=12; while read line; do
[ $ct == 1 ] && echo -n Plus || echo -n $ct;
echo ” $line”;
((ct–));
done < keyboards drumming
... lines removed ...
command line hist-or-y!
EoLines

First we're initializing the "ct" variable we'll be using to count down the 12 Days of Fu. Then we start a while loop to read lines from the standard input.

Within the body of the loop, we use the quick and dirty "[...] && ... || ..." idiom that I've used in previous Episodes as a shortcut instead of a full-blown "if ... then ... else ..." clause. Basically, if we've counted down to one then we want to output the word "Plus"; otherwise we just output the value of $ct. Notice we use "echo -n ..." so that we don't output a newline. This allows us to output the text we've read from the standard input as the remainder of the line. Finally, we decrement $ct and continue reading lines from stdin.

The interesting stuff happens after the loop is over. Yeah, I could have put the text into a file and read the file. But I was looking for a cool command-line way of entering the text. The "EoLines" syntax at the end of the loop starts what's called a "here document". Basically, I'm saying that the text I type in on the following lines should be associated with the standard input (of the while loop in this case). The input ends when the string "EoLines" appears by itself on a line. So all I have to do is type in the 12 lines of text for our 12 Days of Fu and then finish it off with "EoLines". After that we get our output. Neat!

Everybody clear? Cool. I now throw it over to Tim to get the lowdown on his PowerShell madness.
Tim goes back in time

Let's unwrap what we did last week.

PS C:\> $ct=12; “keyboards drumming
admins smiling
… lines removed …
command line hist-or-y!”.split(“`n”) |
% { if ($ct -eq 1) {“Plus $_”} else {“$ct $_”}; $ct– }

Here is the break down:

First, we initialize the $ct variable to begin our count down.

ct=12;

That was easy. Next, we take a multi-line string and split it in to an array using the new line character (`n) as the delimiter.

“keyboards drumming
… lines removed …
command line hist-or-y!”.split(“`n”)

We could have just as easily read in content from a file using Get-Content, but I wanted to demonstrate some new Fu that was similar to Hal’s Fu. The nice thing is that reading from a file would be an easy drop-in replacement.

Once we have an array of Fu Text, we pipe it into ForEach-Object so we can work with each line individually.

if ($ct -eq 1) {“Plus $_”} else {“$ct $_”}

Inside the ForEach-Object loop we use an IF statement to format the output. If the count is one, then we output “Plus” and the Fu Text, otherwise we output the value of $ct and the Fu Text.

Finally, we decrement the value of $ct.

$ct–

Simple right? That was pretty straightforward. But sit back, and grab some spiked Egg Nog before you proceed further.
Ed Surveys the Madcap Mayhem:

I really liked this episode because it required the use of a few techniques that we haven’t yet highlighted in this blog before. Let’s check them out, first reiterating that command:

c:\> copy con TempLines.txt > nul & cmd.exe /v:on /c “set ct=12& for /f
“delims=” %i in (TempLines.txt) do @(if not !ct!==1 (set prefix=!ct!) else (set prefix=
Plus)) & echo !prefix! %i & set /a ct=ct-1>nul” & del TempLines.txt
keyboards drumming
—-snip—-
command line hist-or-y!
^Z (i.e., hit CTRL-Z and then hit Enter)

OK… we start out with the copy command, which of course copies stuff from one file to another. But, we use a reserved name here for our source file: con. That’ll pull information in from the console, line by line, dumping the results into a temporary file, very cleverly named TempLines.txt. Of course, there is the little matter of telling “copy con” when we’re done entering input. While there are several ways to do that, the most efficient way to do so that has minimal impact on the contents of the file is to hit CTRL-Z and then Enter. Voila… we’ve got a file with our input. By the way, I throw away the output of “copy con” with a “> nul” because I didn’t want the ugly “1 file(s) copied.” message to spoil my output. By the way, it kinda stinks that that message is put on Standard Output, doesn’t it? Where I come from, that is much more of a Standard Error thing. But, I’m sure we could get into a big philosophical debate about what should go on Std Out and what should go on Std Err. But, let’s just cut the argument short and say that many Windows command line tools are all screwed up on this front, regardless of your philosophy. That’s because there are no reasonable and consistent rules for directing stuff to Std Out versus Std Err in Windows command line output.

Anyway, back to the point. I then run “cmd.exe /v:on” so I can do delayed variable expansion. That’ll let me use variables with values that can change as my command runs. Otherwise, cmd.exe will expand all variables to their values instantly when I hit Enter. I need to let these puppies float. I use the cmd.exe to execute a command, with the /c option, enclosing the command in double quotes.

So far, so good. But, now is where things get interesting.

To map to Hal and Tim’s fu, I then set a variable called ct to a value of 12, just a simple little integer. But, what’s with that & right after the 12? Well, if you leave it out, you’ll see that the output will contain an extra space in “12 keyboards drumming”… it’ll look like “12[space][space]keyboards drumming”. Where does the extra space come from? Well, check this out:

c:\> cmd.exe /v:on /c “set var=foo & echo !var!bar”
foo bar

c:\> cmd.exe /v:on /c “set var=foo& echo !var!bar”
foobar

Do you see what’s happening here? In the first command, the set command puts everything in the variable called var, starting right after the = and going up to the & (which separates the next command), and that includes the space! We have to omit that space by having [value] with the & right after it. For a more extreme example, consider this:

c:\> cmd.exe /v:on /c “set var=stuff & echo !var!blah”
stuff blah

I typically like to include a space before and after my & command separators when wielding my fu, for ease of readability. However, sometimes that extra space has meaning, so it has to be taken out, especially when used with the set command to assign a string to a variable, like the ct variable in that big command above.

Wait a second… earlier I referred to ct as an integer, and now I’m calling it a string? What gives? Just hold on a second… I’ll come back to that in just a bit.

We have to deal with our FOR loop next. I’m using a routine FOR /F loop to iterate through the contents of the TempLines.txt file. I’m specifying custom delimiters, though, to override the default space and tab delimiters that FOR /F loops like to use. With “delims=”, I’m specifying a delimiter of… what exactly? The equals sign? No… that’s actually part of the syntax of specifying a delimiter. To make an equals a delimiter, I’d have to use “delims==”. So, what is the delimiter I’m setting here? Well, friends, it’s nothing. Yeah. I’m turning off parsing, because I want the full line of my input to be set to my iterator variable. In the past, when I was but a young cmd.exe grasshopper, I would turn off such parsing by setting a delim of a crazy character I would never expect in my input file, such as a ^, with the syntax “delims=^”. But, I now realize that the most effective way to use FOR /F loops is to simply let them use you. I turn off parsing by making a custom delimiter of the empty set. Do not try and bend the spoon… That’s impossible. Instead try to realize the truth…. that there is no spoon.

Anyway, so where was I? Oh yea, with my delimiterless lines now being assigned one by one to my iterator variable of %i, I’m off and running. In the DO clause of my FOR loop, I turn off the display of commands (@) and jump right into an IF statement, to check the value of my ct variable. I expand ct into its value with a !ct!, because I’m using delayed variable expansion. Without delayed expansion, variables are referred to a %var%. My IF statement checks to see if !ct! is NOT equal (==) to 1. If it’s not, I set another variable called prefix to the value of ct.

Then, I get to my ELSE clause. Although I use ELSE a lot in my work, I have to say that this is the first time I’ve had to use it in one of our challenges on this blog. The important thing to remember with ELSE clauses in single commands (not in batch scripts) is that you have to put everything in parentheses. So, if !ct! is equal to 1, my ELSE clause kicks in and sets the prefix variable to the string “Plus”. That way, later on, I can simply print out prefix, which will contain the ct number for most of the days, but the word “Plus” for the last day.

And, here we back to that string/integer thing I alluded to above. The cmd.exe shell uses loosely typed variables. No, this is not a reference to how hard you hit the keys on your keyboard when typing. Instead, like many interpreted languages, variable types (such as strings and integers) are not hard and fast. In cmd.exe, they are evaluated in real time based on context, and they can even change in a single command. My ct variable behaves like an integer, for the most part. I can add to it, subtract from it, and store its value in another variable. But, when I defined it originally with the set command, if I had used “set ct=12 &…”, it would have been a string with the trailing space until I used it in some math, and then that space would disappear. Also, the prefix variable is given the value of ct most of the time, which is just an integer. But, when ct is one, I give the prefix variable a string of “Plus”. I’m an old C-language programmer, so this loose type enforcement kinda weirds me out. Still, it’s quite flexible.

Then, I echo the prefix (!prefix!) and the line of text (%i). I then subtract one from my ct with the “set /a ct=ct-1″, throwing the output of the set command away (>nul). Note that I want to show the prefix and the text on the same line, so I use a single echo command to display both variables on the same line. Most cmd.exe command-line tools actually put their output on standard out with a Carriage Return Line Feed (CRLF) right afterward. Thus, two echo commands, one for each variable, would have broken the prefix and the file content on separate lines, a no-no when trying to reproduce exactly the output of Hal and Tim. When formulating commands that need to display a lot of different items on a single line, I often chunk them into variables and then echo them exactly as I need them on a single line with a single echo statement.

Now, there is one interesting counter-example to the general rule that cmd.exe command-line tools insert a CRLF at the end of their output: the “set /a” command. It does its math, and then displays the output without any extraneous return, as in:

c:\> set /a 8-500 & echo hey
-492hey

I used that little fact in this fun command to spew Matrix-like gibberish on the screen from Episode #58:

C:\> cmd.exe /v:on /c “for /L %i in (1,0,2) do @set /a !random!”

When I first was working on this 12-days challenge, I was thinking about using set /a to display !ct! and then the line from the file. It would all be on the same line because of that special “no-CRLF” property of “set /a”. But, I ran into the little problem of the “Plus” for the last line of input, so I instead introduced the prefix variable and played on the loose typing. There are dozens of other ways to do this, but I tried to focus on one that, believe it or not, I thought made the most sense and was simplest.

Oh, and to close things out, I delete the TempLines.txt file. Can’t litter our file system with crap, now can we?

So, as you can see, there were a bunch of ideas we haven’t used in this blog so far that popped out of cmd.exe in this innocuous-seeming challenge, including empty-set delims, an ELSE clause, weak type enforcement, and variable building for a single line of output. That’s a lot of holiday cheer, and it makes me happy.

With that said, all of us at the Command Line Kung Fu blog would like to wish our readers a Happy and Prosperous New Year!

admin | Dec 30, 2009 | Comments 0

Yule Love It!

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal has indulged in a bit too much holiday cheer:

Presented for your enjoyment with no explanation or excuses:

$ ct=12; while read line; do [ $ct == 1 ] && echo -n Plus || echo -n $ct; echo " $line"; ((ct--)); done <<EoLines keyboards drumming admins smiling systems thrashing networks crashing hosts a-pinging Windows versions (billion) Linux distros Windows loops! authors coding shells hacked types of hosts command line hist-or-y! EoLines 12 keyboards drumming 11 admins smiling 10 systems thrashing 9 networks crashing 8 hosts a-pinging 7 Windows versions 6 (billion) Linux distros 5 Windows loops! 4 authors coding 3 shells hacked 2 types of hosts Plus command line hist-or-y!

Tim got run over by a reindeer:

PS C:\> $ct=12; "keyboards drumming admins smiling systems thrashing networks crashing hosts a-pinging Windows versions (billion) Linux distros Windows loops! authors coding shells hacked types of hosts command line hist-or-y!".split("`n") | % { if ($ct -eq 1) {"Plus $_"} else {"$ct $_"}; $ct-- } 12 keyboards drumming 11 admins smiling 10 systems thrashing 9 networks crashing 8 hosts a-pinging 7 Windows versions 6 (billion) Linux distros 5 Windows loops! 4 authors coding 3 shells hacked 2 types of hosts Plus command line hist-or-y!

Ed’s Nuts Roasting on an Open Fire:

c:\> copy con TempLines.txt > nul & cmd.exe /v:on /c "set ct=12& for /f "delims=" %i in (TempLines.txt) do @(if not !ct!==1 (set prefix=!ct!) else (set prefix= Plus)) & echo !prefix! %i & set /a ct=ct-1>nul" & del TempLines.txt keyboards drumming admins smiling systems thrashing networks crashing hosts a-pinging Windows versions (billion) Linux distros Windows loops! authors coding shells hacked types of hosts command line hist-or-y! ^Z (i.e., hit CTRL-Z and then hit Enter)

12 keyboards drumming 11 admins smiling 10 systems thrashing 9 networks crashing 8 hosts a-pinging 7 Windows versions 6 (billion) Linux distros 5 Windows loops! 4 authors coding 3 shells hacked 2 types of hosts Plus command line hist-or-y!
Best wishes for a happy holiday season and a joyous and prosperous new year!

admin | Dec 22, 2009 | Comments 0

Getting the perfect Perm(s)

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Tim unwraps:

One of the things I find myself doing on a regular basis is creating a new directory structure and setting the permissions. The permissions are different for each folder and are based on who in the organization needs access to it. We could just write a script to create the directories and the permissions, but let’s say we want to copy permissions from one directory structure to another. For this example let’s assume we have a project folder structure that looks something this.

Prjs +-Project1 (Managers - Full Access, Consultants - Full Access) |-Budget (Consultants - Deny, Finance - Full Access) |-Data +-Docs |-ForRelease (AdminStaff - Full Access) +-InProgress
Included above is the appropriate permissions on each folder. All permissions are inherited, so consultants and managers would have access to the Data directory.

We can verify these permissions by using Get-ChildItem (aliases gci, dir, ls) and piping the results into Get-Acl.

PS C:\> ls Prjs -recurse | Get-Acl | fl Path,AccessToString

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1 AccessToString : WINXP\Consultants Allow FullControl WINXP\Managers Allow FullControl

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Budget AccessToString : WINXP\Consultants Deny DeleteSubdirectoriesAndFiles, Modify, ChangePermissions, TakeOwnership WINXP\Consultants Allow FullControl WINXP\Finance Allow FullControl WINXP\Managers Allow FullControl

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Data AccessToString : WINXP\Consultants Allow FullControl WINXP\Managers Allow FullControl

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Docs AccessToString : WINXP\Consultants Allow FullControl WINXP\Managers Allow FullControl
Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Docs\ForRelease AccessToString : WINXP\AdminStaff Allow FullControl WINXP\Consultants Allow FullControl WINXP\Managers Allow FullControl

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Docs\Working AccessToString : WINXP\Consultants Allow FullControl WINXP\Managers Allow FullControl

So now we want to create a second project, Project2, and we want to make sure we have the same permissions. We could copy just the directories without files, but there may be more subdirectories further down that we don’t want. So let’s create the folders.

PS C:\> mkdir Prjs\Project2\Budget PS C:\> mkdir Prjs\Project2\Data PS C:\> mkdir Prjs\Project2\Docs\ForRelease PS C:\> mkdir Prjs\Project2\Docs\Working

Note, when the Budget directory is created it also creates the Project2 directory since it doesn’t exist.

What are the permissions on the new folder?

PS C:\Prjs> Get-Acl Project2 | Get-Acl | fl Path,AccessToString

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project2 AccessToString : BUILTIN\Administrators Allow FullControl NT AUTHORITY\SYSTEM Allow FullControl WINXP\myuser Allow FullControl CREATOR OWNER Allow 268435456 BUILTIN\Users Allow ReadAndExecute, Synchronize BUILTIN\Users Allow AppendData BUILTIN\Users Allow CreateFiles
Those are not the permissions we want. The permissions need to be copied from Project1 to Project2, but how? The Get-Acl and Set-Acl commands will do it.

PS C:\Prjs> Get-Acl Project1 | Set-Acl Project2

Let’s verify.

PS C:\Prjs> Get-Acl Project2 | Get-Acl | fl Path,AccessToString

Path : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project2 AccessToString : BUILTIN\Administrators Allow FullControl WINXP\Managers Allow FullControl WINXP\Consultants Allow FullControl

Looks good. Now the subfolder permissions need to be copied as well.

PS C:\Prjs> ls Project1 -Recurse | Get-Acl |% { Set-Acl $_ -Path ($_.Path -replace "Project1","Project2") }
First we do a recursive directory listing and get the Acl on each folder. We then take that Acl and apply it to a different folder. In our case all we need to do is replace Project1 for Project2 in the Path. Let’s verify that the permissions match.

PS C:\Prjs> Compare-Object (ls Project1 -Recurse | Get-Acl) (ls Project2 -Recurse | Get-Acl) -Property PSChildName, Access

No output, that’s good, it means the permissions are identical. How did that work?

The Compare-Object cmdlet is used to find the differences between the collection of objects returned by these two commands:

ls Project1 -Recurse | Get-Acl ls Project2 -Recurse | Get-Acl
The Property parameter specified in the original command allows us to select the properties to be checked for differences. PSChildName is the directory name and the Access property contains the permissions on the folder. We can’t substitute the Path property for PSChildName since Path is the full path and it would always be different.

Copying permissions is pretty easy, I imagine it will be pretty easy for Hal since it isn’t as granular. Finally, a bit of a leg up on Hal.

Hal just copies everything:

Do I detect a trace of jealousy and bitterness in my colleague’s last comments? Better fix up that attitude Tim, or there will be nothing but coal in your stocking this year.

It’s interesting that Tim brings up this subject, because it’s another case where the differences in philosophy between Windows and Unix are apparent. In Windows, you need to fix up your directory permissions with an external tool after you copy the files. In Unix, it’s just a natural part of the file copying operation– particularly if you’re doing the copy as the superuser.

This is also an area where we’ve seen some historical evolution in Unix-like operating systems. When I first got started with Unix in the 1980’s, the “cp” command didn’t have a “-p” option to preserve permissions, ownerships, and timestamps. The way you would copy directories when you wanted to preserve directory permissions was with the so-called “tar cp” idiom (actually, real old-timers will remember doing this with cpio):

# cd olddir # tar cf - . | (cd /path/to/newdir; tar xfp -)
Here we’re running the first tar command to create (“c”) a new archive from the current working directory (“.”) and write it to the standard output (“f -”). We pipe that output to a subshell that first changes directories to our target dir and then runs another tar command to unpack the incoming archive on the standard input. The “p” option means preserve permissions, timestamps, and ownerships. Actually “p” is normally the default if you’re running the tar command as root, so you can leave it off, but I prefer being explicit.

These days, however, there are a couple of simpler options. Obviously, you could just use “cp -p”:

# cp -Rp olddir /path/to/newdir

I generally prefer rsync though:

# rsync -aH olddir /path/to/newdir

rsync not only allows you to copy directories within the same system, but also gives you the option of copying directories across the network. Also, if you just want to update the permissions on a directory, the rsync command will do that and not actually copy any file data that has previously been copied. For more information on rsync, see Episode #24.

One issue that Tim brought up was that sometimes you want to copy only part of a directory structure, but exclude certain files and/or subdirectories. This is another place where rsync beats cp. rsync has a couple of different ways of excluding content: the –exclude option for specifying patterns to exclude on the command line, and –exclude-from for reading in a list of patterns to exclude from a file. There’s no way of excluding files built into the cp command at all. For those old fogies like me out there who still occasionally use “tar cp”, the tar command typically has a switch like -X to exclude files and directories from the archive, and GNU tar has –exclude options very similar to rsync.

One thing you do need to be careful with for all of these copy options, however, is that they may not copy special permissions like extended attributes or file ACLs by default. Both cp and rsync have explicit options you can set to preserve these settings:

# cp -R --preserve=all olddir /path/to/newdir # rsync -aHAX olddir /path/to/newdir

There’s no way to do something similar with the “tar cp” idiom, because the tar archive format doesn’t preserve extended attributes and ACLs.

Oh dear. Now it’s Ed’s turn. I hope Tim and I haven’t spoiled his holiday cheer…

Ed Joyously Responds:

Ahhhh…. file permissions. They tend to be an absolute pain in the neck to deal with en masse in cmd.exe. Sure, we can use cacls or icacls to manipulate them on single files or directories just swell. But, synchronizing or applying changes to bunches of files using cacls or icacls is often dangerous and painful. When I first read Tim’s challenge, I thought to myself, “This is gonna get ugly on us… as ugly as that feud between Snow Miser and Heat Miser.” I immediately began to search my mind for a hook or trick to make this a lot easier, hoping to avoid a trip to visit Mother Nature.

Then, it hit me: we can use our little friend robocopy, the wonderfully named tool in Vista, Windows 7, and Windows 2008! Yeah, it’s not built in to XP or Windows 2003, but it’ll work for the latest version of Windows. We talked about robocopy in Episode #24.

To address Tim’s challenge, I’m going to assume that the directory structure where we want to replicate our file permissions does not already exist, avoiding the mkdir commands Tim uses. Robocopy will make those for us, dutifully placing the proper access controls on them if we run:

C:\> robocopy [directory1] [directory2] /e /xf *

All of the subdirectories in directory1 will be created in directory2 with the same file permissions. The /e will make it recurse through those subdirectories, copying both directories with stuff in them and empty directories. The /xf means I want to exclude a certain set of files, which I’ve selected as *, meaning to exclude all files — Only directories will be copied, including all of their luscious permissions.

Well, that’s all fine and good, but what about Windows XP and 2003? Well, you can download and install robocopy on either of them, which is a pretty good idea. Alternatively, there is a way to trick Windows into applying the permissions from one directory structure to another, which applies to Windows 2003, Vista, 7, and 2008 Server. For this trick, we’ll use the handy /save and /restore feature of icacls. Here, let’s follow Tim’s lead, and assume that we’ve got directory1 and directory2 already created, and we want to take the permissions from directory1 and its subdirectories and apply them to the already-existing directory2. Check out the following command:

C:\> icacls [directory1]\* /save aclfile /t /c

This command tells Windows to run icacls against directory1 and all of its contents (*), saving the results (/save) in a file called aclfile, recursing through the directory structure (/t), not stopping when it hits a problem (/c). Now, the resulting aclfile is not regular ASCII, but instead a unicode format that includes all of the permissions for the directories _and_ files inside of directory1.

Now, if there is a directory2 that already exists and has a similar overall structure to directory1, but perhaps without having any files in it, we can use icacls to restore the aclfile on a different directory! Wherever there is commonality in the directory structure, the permissions from directory1 will be used to overwrite the permissions on the given entity in directory2. The command to use is:

C:\> icacls [directory2] /restore aclfile /t /c

Voila! We’ve restored the ACLs from directory1 onto directory2! Now, that is a delicious holiday treat.

But, that leaves out poor little Windows XP, an operating system without robocopy and icacls built in. Sad, sad, sad little XP. Looks like it gets a lump of coal in its stocking this year, not only from Santa-Ed, but also from Microsoft, which has announced its impending withdrawal of support of this very near and dear friend.

admin | Dec 15, 2009 | Comments 0

The Tangled Web

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal gets a soft one this week

Lately we’ve had some of our loyal readers– mostly the Windows folk– asking about command-line tools for accessing web pages. When these questions come up, I just smile serenely, because it’s easy to do this in Unix. Ed and Tim on the other hand turn a little green and start sweating.

Among the Unix tribes, there seem to be two popular command-line tools for grabbing web pages. Some folks prefer curl, but I generally stick with wget since it’s the tool I learned first. Both curl and wget support HTTP, HTTPS, and FTP as well as proxies of various types along with different forms of authentication. curl also supports other protocols like TFTP, FTPS (FTP over SSL), and SCP and SFTP.

Using wget couldn’t be simpler:

$ wget blog.commandlinekungfu.com --2009-11-18 18:47:51-- http://blog.commandlinekungfu.com/ Resolving blog.commandlinekungfu.com... 74.125.113.121 Connecting to blog.commandlinekungfu.com|74.125.113.121|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: `index.html'

[ <=> ] 173,805 18.3K/s in 9.3s
2009-11-18 18:48:01 (18.3 KB/s) - `index.html' saved [173805]

Or, if you don’t like all the noisy chatter, you can use the “-q” option:

$ wget -q blog.commandlinekungfu.com $ ls index.html index.html.1

Notice that wget doesn’t overwrite the first index.html file we downloaded. Instead it appends a uniqe number to the file name. If we downloaded another index.html file, it would end up as index.html.2 and so on. You can also use “-O” to specify an output file name.

Maybe my favorite feature, however, is the “-r” (recursive) option that allows me to grab an entire tree of content in a single operation. So all I have to do when I want to back up the content here at Command Line Kung Fu is just run “wget -r -q blog.commandlinekungfu.com” and wait a few minutes for everything to download.

There are lots of other neat features to wget, but I think I’ve already demoralized my Windows brethren enough for one Episode. Let’s see what Ed and Tim cook up.

Tim orders take-out:

Unfortunately (again), there isn’t a built-in cmdlet to do the equivalent of wget or curl, but we can access the .NET libraries and recreate some of the functionality of the Linux commands. By default, the .NET Framework supports URIs that begin with http:, https:, ftp:, and file: scheme identifiers, so it isn’t quite as full featured as Linux, but it is all we have.

PS C:\> (New-Object System.Net.WebClient).DownloadString( "http://blog.commandlinekungfu.com") ...
This will grab files in text format and it can be used further down the pipeline, such as saving the file by piping it in to Out-File. What if we want to grab non-text files or just download the file? We can use the DownloadFile method and specify where we want to save the file.

PS C:\> (New-Object System.Net.WebClient).DownloadFile( "http://blog.commandlinekungfu.com","c:\downloadedfile.html")
What happens if the file doesn’t exist? It raises an 404 error and the file (obviously) isn’t opened.

PS C:\> (New-Object System.Net.WebClient).DownloadString( "http://blog.commandlinekungfu.com/NonExistant.file")

Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (404) Not Found." At line:1 char:49 + (New-Object System.Net.WebClient).DownloadString( <<<< "http://blog.commandlinekungfu.com/NonExistant.file")

Sadly, there isn’t way to recursive requests without doing some heavy duty scripting.

Ed Sits Here Trying Not To Think About What Hal Called His Part of this Article:

Ahhh… an age-old question. I remember about a year ago, I was having a discussion with Kevin Johnson, bon vivant and web attacker extraordinaire, when this issue came up. I posed the following question to KJ0 (as we call him): “Suppose you’ve just hacked a Windows box in a pen test. You have command shell access of said machine. Tell me what you want to do now, using only built in command-line capabilities.” I went further, boldly mixing my metaphors: “Consider me as /dev/WindowsCommandLine… where do you want to go today?”

Kevin shrugged, smiled, and said, “Easy… wget.”

I felt a piercing pain stab at my heart. “The wget tool has sooo many options… can we scale it back a bit? What do you really want to do?” Kevin said, “Sure… I just wanna fetch a web page and write it to the file system. How can I do that at the command line in Windows?”

I spent a little time fitzing around with various Windows commands, and ultimately settled on using the built-in Windows telnet client to formulate HTTP requests manually. It’s crude, but works.

“But, Ed,” you argue, “Microsoft removed the telnet client from Windows Vista and some of the 6 million versions of Windows 7.” I respond: Yes, it’s no longer installed by default, but on the professional versions of Vista and 7, you can run:

C:\> start /w pkgmgr /iu:"TelnetClient"

Even if it doesn’t finish, kill that cmd.exe, and you should have the Telnet Client ready to run. Note that the installation package for the telnet client is built-in, but the tool itself just isn’t installed. The machine doesn’t have to reach out to the Internet to get it. Also, note that /iu stands for install update, you need that :, and you should observe the case of the Cap-T and Cap-C in TelnetClient. Oh, and by the way… if you want the Telnet service, change “TelnetClient” to “TelnetServer”. To remove either after you’ve installed, it, run the same command, except substitute /uu: (uninstall update) for /iu:.

So, now equipped with the telnet client, let’s make an HTTP request.

Now, you might think that we should simply echo an HTTP request and pipe it into the telnet client, right? Bzzzzzt. Sorry, my friend, but the Windows telnet client doesn’t like anything to come into its Standard Input at the command line, and it certainly doesn’t send it to the target machine. Try sniffing it sometime. Another annoying part about the Windows telnet client is that it doesn’t like you to touch its Standard Output. The telnet client is somehow clairvoyant, and if you follow it with a > or a |, it knows and refuses to send any packets. Gee, thanks, Microsoft.

For this one, we’re going to have to go interactive.

C:\> telnet -f log.txt

I’m logging output to the file log.txt, where our output will reside.

At our new telnet prompt, we can open a connection to port 80 on our destination web server. We’ll use the “o” option to open a connection:

Microsoft Telnet> o www.google.com 80 Connecting To www.google.com...

Now, depending on the version of Windows you are using, it may or may not look like a connection is made. Either way, just trust that it has, and start typing your HTTP request. Typically, you’ll be getting a page, which you can do by entering:

GET [pagename] HTTP/1.1 Host: [TheHostName]

Hit Enter Enter at the end, and you’ll have the page displayed on your screen. Hit Enter again, and you’ll get back to your telnet prompt. Type “quit” to exit.

More importantly, you should now have your page in the output file log.txt. Yes, you’ll have the HTTP response header in front of it, but that’s not so bad. It’s ugly, but it’ll let you grab a file quickly from a server.

admin | Nov 24, 2009 | Comments 0

Help Fu

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Tim hits lead off:

This week we get help with help. Everyone needs help, and some in different ways than others. But we aren’t going to discuss Ed’s problem with crunchy peanut butter or Hal’s issue with furry animals with tails. Anyway, back to the issue at hand.

Instead of jumping right in to getting help on commands, let’s use PowerShell to find commands by using the Get-Command cmdlet. This command will retrieve all the cmdlets, aliases, and functions available to us. In order to use this command effectively, let’s go over a bit of the very basics of PowerShell. As you all probably know, the nomenclature for PowerShell cmdlets is verb-noun. Microsoft’s approved verb list defines the verbs so you know which verb to use and to ensure consistent use by cmdlet developers. That way the verbs are consistent so “get” is always used instead of a random choice of get, retrieve, grab, or obtain. The noun part of the name identifies the entity on which the action is performed.

Ed and Hal discussed the command history in episode #27, so we’ll use that as the focus of our research.

Let’s see a list of all the available nouns.

PS C:\> Get-Command -Type cmdlet | Sort-Object noun | Group-Object noun

Count Name Group ----- ---- ----- 2 Acl {Set-Acl, Get-Acl} 5 Alias {Import-Alias, Get-Alias, Set-Alias, Export-... ... 4 History {Invoke-History, Clear-History, Get-History,... ...

You probably could have already guessed, but the noun we want to use is “History.” To get a lits of the commands available using that noun we can use this command:

PS C:\> Get-Command -Noun history

CommandType Name Definition ----------- ---- ---------- Cmdlet Add-History Add-History [[-InputObject] ... Cmdlet Clear-History Clear-History [[-Id] Cmdlet Get-History Get-History [[-Id] ... Cmdlet Invoke-History Invoke-History [[-Id] Get-Command -Verb Get

Back to the subject at hand…

Now we have a list of cmdlets we are interested in. Let’s see how we can use the command by using the Get-Help cmdlet.

PS C:\> Get-Help Get-History

NAME Get-History

SYNOPSIS Gets a list of the commands entered during the current session.

SYNTAX Get-History [[-Id] ] [[-Count] ] [] ...
We have a bit of a problem here, the basic help isn’t super useful since it doesn’t provide a good description of the parameters or any examples of the cmdlet’s use. However, it does provide syntax and a list of parameters. To see the full help including examples and parameter descriptions use this command:

PS C:\> Get-Help Get-History -Full

To see just the examples:

PS C:\> Get-Help Get-History -Examples

As you have probably noticed in past episodes there are a lot of aliases available in PowerShell. The Get-Alias cmdlet can be used to “get” the list of aliases. Specifically, we can see the aliases for the commands using the History noun.

PS C:\> Get-Alias | Where-Object {$_.Definition -like "*history"}

CommandType Name Definition ----------- ---- ---------- Alias clhy Clear-History Alias ghy Get-History Alias h Get-History Alias history Get-History Alias ihy Invoke-History Alias r Invoke-History

The Get-Command cmdlet can be used to get the same results.

PS C:\> Get-Command -Type Alias | Where-Object {$_.Definition -like "*history"}

That’s about it for PowerShell, let’s see what Ed has for us with the Windows command line: Get-Help -FuMaster Ed -Subject Windows

Ed’s Output:

You guys with your fancy help capabilities in your shells are a real hoot. Yes, your shells include intuitive and easy-to-access help features. Where’s the fun in that? Or, more to the point, where’s the fu in that? I’ve spent many countless hours sitting at a cmd.exe trying to reverse engineer… uh… I mean conducting detailed design recovery of cmd.exe functionality to try to figure out what the heck the little bugger was really doing. Ah… good times.

In all seriousness though, cmd.exe offers an embarrassingly paltry amount of help for using it and its commands. Really, it’s kind of stinky. Let’s explore the few skimpy options we have.

First off, we can run the help command, followed by the command we want to analyze, as in:

C:\> help findstr

The good news? Well, we’ll see a brief description of findstr functionality, as well as its command flags.

The bad news? Well, where to start? How about:

Bad News #1) The help command doesn’t provide help for all of the commands available in cmd.exe. It only covers a little less than 80 of them.

To get a list of the commands that help has information about, simply run:

C:\> help

Some of the most interesting and useful commands of all, like netstat, netsh, reg, sc, and wmic, result in this unhappy little message:

C:\> help reg This command is not supported by the help utility. Try "x /?".

Bad News #2) The output of the help command doesn’t include any examples or much detail at all. It’s just really basic usage, without any interesting fu included.

Bad News #3) Note the little suggestion in the output of my command in Bad News #1: Windows tells us to try running the command with a /? argument for help. As an old Unix guy (shhhh… don’t tell anyone), I’m very hesitant to run a command that I don’t understand just so I can get help. It scares me to do so without knowing what just running the command might do. And, here’s a dirty little secret of the help command itself. It actually runs the command you are trying to get help about! Yes, check this out…

First, start Task Manager, move to the Processes tab, and sort it alphabetically by clicking on the Image Name column.

Now, in a separate cmd.exe, run the following command, which will get help for the findstr command, repeatedly:

C:\> for /L %i in (1,0,2) do @help findstr

Look in your Task Manager window… see it? Every once in a while, the findstr command pops up! So “help findstr” is actually invoking the findstr command itself to scrape out its help options. Yowza. That’s just painful for me.

Bad News #4: Well, with only around 80 commands covered by the help command, we often have to turn to that old stand by, “[command] /?” for more help. But the help included here is spotty at best as well, with major inconsistencies between commands and some commands supporting context-specific help for particular sub-areas of the command. For example, “wmic /?” does offer some help, but there is a wmic-specific option for getting more detailed help:

C:\> wmic /?:full

Besides wmic, none of the other commands I know about support this /?:full option.

Also, some commands have specific help for certain contexts within the command. Consider the following examples:

C:\> wmic process get /?

This shows you a list of all the process attributes wmic supports.

Or, try this:

C:\> wmic process call /?

This one shows you a list of all the process methods you can call from within wmic.

Or, how about this one:

C:\> net /?

And then:

C:\> net use /?

Likewise, we can run “reg /?” to get an overview of the reg command, followed by a “reg query /?” to get more details about the specific syntax for querying the registry.

Want another bewildering inconsistency? Try running this:

C:\> netstat /?

Note that the output actually shows the command flags with dashes instead of slashes (i.e., “netstat –na” instead of “netstat /na”). Thankfully, the actual netstat command on modern Windows boxes lets you use dashes or slashes in its flags.

So, how can we wrap our heads around all of this inconsistency? I’ve found that mastering Linux involves learning certain general principles and then seeing how they are applied throughout the operating system. Whereas, in Windows, mastering the command line involves memorizing a bunch of complex, counter intuitive, and often inconsistent options scattered throughout the operating system without rhyme or reason. So, why bother? Because, once mastered, the Windows command-line is incredibly useful in analyzing systems at a fine-grained level. Also, cmd.exe is pretty much everywhere, pre-installed, so there’s a lot less hit-and-miss than you get with installs of PowerShell. At least, there is for now… this is changing as PowerShell gets more widely deployed.

The bottom line here? When I’m actually looking for help with a command, I apply the following process. I start by trying “help [command]”. If that provides no satisfaction, I proceed to “[command] /?”. If that doesn’t give me what I want, I try to look for context-specific help with “[command] [option] /?”. And, if that doesn’t get me where I need to go, I turn to the web to research commands. One of my absolutely favorite sites for researching Windows (as well as Linux and Unix) commands is the awesome ss64 site. It includes a detailed list of most Windows commands, including their various flags, example usage, and mapping to rough equivalents in Linux, OS X (sometimes), and PowerShell (occasionally). That’s awesome. And, finally, there’s Microsoft’s own command line reference, worth a check from time to time.

Hal cleans up:

There are actually a number of different help systems available on a typical Unix system. First there’s the standard on-line manual pages accessed via the “man” command. The Unix manual pages include not only information about programs that you can run from the command line, but also documentation on programming APIs, device interfaces, and even the format of important configuration files on the system.

The trick sometimes is getting to the right manual page. For example, suppose I wanted some information on how to use the chmod() system call in my C program. If I just typed “man chmod”, I’d get the manual page for the chmod program and not the documentation on the system call. To distinguish these two different manual pages, the Unix manual has traditionally been organized into sections. Sections 1 (user commands) and 8 (administrator commands) are devoted to command-line tools, while sections 2 (system calls) and 3 (library routines) are devoted to programming APIs. FYI, section 4 is device interfaces, section 5 is configuration file formats, section 6 is games, and section 7 is “miscellaneous” but includes useful tidbits like the table of ASCII values (“man ascii”) and additional details on various IPC mechanisms. When a Unix document refers to “chmod(2)” it means “the documentation on the chmod system call in section 2 of the manual”.

But how does on pull up the chmod(2) manual page instead of the default chmod(1) page? The man command takes a “-s” option to specify the section: “man -s 2 chmod”. But on many Unix variants you can simply drop the “-s” and just type “man 2 chmod”.

Suppose, however, that I didn’t know which section the chmod() system call was documented in. One of the most useful features of the man command is the “-k” (keyword search) option:

$ man -k chmod chmod (1) - change file mode bits chmod (2) - change permissions of a file fchmod (2) - change permissions of a file fchmodat (2) - change permissions of a file relative to a directory f...

You don’t have to search on command names, of course. Any related keyword will work. For example, if you’re new to Unix and don’t know how to rename files:

$ man -k rename dpkg-name (1) - rename Debian packages to full package names lvrename (8) - rename a logical volume mmove (1) - move or rename an MSDOS file or subdirectory mren (1) - rename an existing MSDOS file mv (1) - move (rename) files prename (1) - renames multiple files rename (1) - renames multiple files rename (2) - change the name or location of a file rename.ul (1) - Rename files renameat (2) - rename a file relative to directory file descriptors vgrename (8) - rename a volume group

By the way, the “apropos” and “whatis” commands are equivalent to “man -k”. However, all of these commands operate on special “databases” of information that has been extracted from the manual pages themselves via the “makewhatis” command (aka “mandb” on Linux). Very often your system will have a cron job that builds the whatis databases automatically every week or so, but on some Unix systems you have to build the databases manually (or create your own cron job).

Another tricky issue with the Unix on-line manual is that sometimes you have multiple repositories of manual pages. For example, you might have the standard OS manual pages under /usr/share/man and manual pages for third-party software in /usr/local/man. The man command lets you use the “-M” option to specify a path to use for finding manual pages, but far it’s far easier to set MANPATH in your .bashrc file:

export MANPATH=/usr/share/man:/usr/local/man

MANPATH works just like PATH or LD_LIBRARY_PATH. In the above example, the man command will check the system manual page directory first and then look in /usr/local/man if it doesn’t find the manual page you’re requesting.

The man command is also the source of one of the most famous jokes in Unix:

$ man 'furry animals with tails' No manual entry for furry animals with tails

OK, so the traditional joke is “man rear”, but you get the idea.

Sadly, man pages seem to be dieing off. A lot of Open Source projects have either very scanty or completely non-existent manual pages. It’s actually pretty tough to write a good manual page, both in terms of content and also because you need to know nroff/troff formatting commands to format the text. But writing (and reading) manual pages properly is still an important Unix skill IMHO.

Another source of on-line documentation on Unix is the Free Software Foundation’s “info” system. The most complete documentation on packages like emacs, gcc, GNU tar, et al is found in the info system– the manual pages are mostly just stubs extracted from the docs in the info pages. Run the command “info info” to get more information on how to navigate the curses-based info interface. Frankly, I think the user interface for the info system is terrible, but it was mostly designed to be run from within emacs. I suppose that it makes sense if you’re one of those people who essentially lives inside of emacs and never touches an actual terminal window.

But the info system never really caught on outside of the FSF. With traditional manual pages being marginalized as well, the main documentation interface these days seems to be the “-h” or “–help” options supported by many commands. Typically, one or both of these options will generate at least a terse summary of command options:

$ info --help Usage: info [OPTION]... [MENU-ITEM...]
Read documentation in Info format.

Options: --apropos=STRING look up STRING in all indices of all manuals. -d, --directory=DIR add DIR to INFOPATH. --dribble=FILENAME remember user keystrokes in FILENAME. -f, --file=FILENAME specify Info file to visit. -h, --help display this help and exit. --index-search=STRING go to node pointed by index entry STRING. -n, --node=NODENAME specify nodes in first visited Info file. -o, --output=FILENAME output selected nodes to FILENAME. -R, --raw-escapes output "raw" ANSI escapes (default). --no-raw-escapes output escapes as literal text. --restore=FILENAME read initial keystrokes from FILENAME. -O, --show-options, --usage go to command-line options node. --subnodes recursively output menu items. -w, --where, --location print physical location of Info file. --vi-keys use vi-like and less-like key bindings. --version display version information and exit.

The first non-option argument, if present, is the menu entry to start from;
it is searched for in all `dir’ files along INFOPATH.
If it is not present, info merges all `dir’ files and shows the result.
Any remaining arguments are treated as the names of menu
items relative to the initial node visited.

Examples: info show top-level dir menu info emacs start at emacs node from top-level dir info emacs buffers start at buffers node within emacs manual info --show-options emacs start at node with emacs' command line options info --subnodes -o out.txt emacs dump entire manual to out.txt info -f ./foo.info show file ./foo.info, not searching dir

Email bug reports to bug-texinfo@gnu.org,
general questions and discussion to help-texinfo@gnu.org.
Texinfo home page: http://www.gnu.org/software/texinfo/

Not as useful as a traditional manual page, IMHO, but at least it’s something. The biggest problem is that with the documentation being tied up in each individual command’s “–help” output, there’s no “man -k” equivalent for doing global keyword searches if you’re not sure what command you’re looking for.

admin | Nov 10, 2009 | Comments 0

Time Lords

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal is still adjusting:

Mr. Bucket suggested a time-related Episode this week in honor of the shift back to Standard Time here in the U.S. Generally, we try to humor Mr. Bucket as long as his requests aren’t too deviant (or if Ed is the only one who has to dress up), so here goes.

First I’ve got a couple of tricks up my sleeve related to the NTP time synchronization service. My favorite command-line tool for checking up on my NTP servers is “ntpdc -p”:

$ ntpdc -pn server.example.com remote local st poll reach delay offset disp ======================================================================= =216.14.97.75 192.168.1.2 16 1024 0 0.00000 0.000000 0.00000 =173.45.227.99 192.168.1.2 2 1024 377 0.06984 -0.003889 0.13681 *64.73.32.135 192.168.1.2 2 1024 377 0.09087 0.002323 0.12178 =127.127.1.1 127.0.0.1 5 64 377 0.00000 0.000000 0.03093

The “st” column that shows the remote server’s stratum level. This is useful for detecting downed time servers, because they show up as stratum 16 like on the first line above. The “poll” (polling interval in seconds) and “reach” (displayed in octal notation and capped at 255, the number of recent successful polls) columns give you and idea of how well your server is synchronizing with other time servers. “delay” tells you how far away the remote server is from you, and “offset” is the difference between your local clock and the clock on the remote server (if things are working right, the offset should be less than a hundredth of a second). Note that in the above example, I checked on the remote server.example.com machine, but you can leave off the host name portion to check the time server running on your local box.

In addition to checking status with ntpdc, you can also set the time on your box from the command line using ntpdate. Normally you would just run ntpdate and specify the server you wanted to synchronize against, but here’s a cute little bit of awk fu for pulling the server names out of your current ntp.conf file:

# ntpdate `awk '/^(server|peer) / && !/127.127./ {print $2}' /etc/ntp.conf` 29 Oct 19:42:43 ntpdate[13376]: adjust time server 66.187.233.4 offset -0.004818 sec

Here I’m pulling off the host name or IP address specified on the “server” or “peer” lines in your ntp.conf file. I’m skipping any “addresses” that start with “127.127.” since these are merely place holders for clock driver definitions and not real IP addresses.

Note that you can only run ntpdate if your NTP daemon is not currently active. If ntpd is running at the same time you execute ntpdate, you get a message like this:

# ntpdate `awk '/^(server|peer) / && !/127.127/ {print $2}' /etc/ntp.conf` 29 Oct 19:42:18 ntpdate[13362]: the NTP socket is in use, exiting

Typically you only need to call NTP date by hand if the NTP daemon is not running and your clock has gotten out of synch. After you use ntpdate to jump your clock to something approximating the right time, you should start your NTP daemon to keep it synced (usually “/etc/init.d/ntpd start”).

Getting away from NTP now, let me show you a little environment variable hack that’s particularly useful for Forensic Analysts. Suppose you’ve captured a disk image from a system that was being used in a time zone other than the one your analysis machine is set to. You’d like to mount a copy of the image and be able to observe the timestamps on the files relative to the time zone the image was taken from. You could change the global time zone setting on your analysis workstation, but that could cause you lots of hassle.

It turns out that bash allows you to set the TZ environment variable in your shell to alter the local time zone setting for only that shell. For example, my local time zone is US/Pacific, but suppose I had an image from a machine from the East Coast:

$ date Thu Oct 29 17:59:13 PDT 2009 $ ls -ld /mnt/image/tmp drwxrwxrwt 23 root root 4096 Oct 29 17:28 /mnt/image/tmp $ export TZ=US/Eastern $ date Thu Oct 29 21:00:07 EDT 2009 $ ls -ld /mnt/image/tmp drwxrwxrwt 23 root root 4096 Oct 29 20:28 /mnt/image/tmp

Cool! Now I can look at the dates on files in my image and not constantly have to be doing mental arithmetic to convert the time stamps.

So there are three of my favorite time-related hacks. If we can get Ed out of those leather chaps that Mr. Bucket had him wear, I’m sure he’ll have some cool Windows Fu to show us…

Ed chimes in:

Ahhh… time. A great idea for an article, Mr. Bucket. Much better than your last idea… you know… the one about the chaps. Anyway, I digress.

Let’s walk before we run. To check the time on your local computer, you can simply run the following command:

C:\> time /t 06:05 AM

Want more precision? We can get that, as discussed in Episode #49, by displaying the %time% environment variable.

C:\> echo %time% 6:05:22.75

Alternatively, the “net time” command can be used to pull the time from a remote (or even local) Windows box:

C:\> net time \\[computername] Current time at \\FRED2 is 10/31/2009 6:04 AM The command completed successfully.
Note that, depending on whether you are a member of a domain and the privileges of your current account, you may need to first initiate an SMB connection with that machine, by running:

C:\> net use \\[computername] [password] /u:[username] The command completed successfully.

If you happen to trust that server, you can set your local machine’s time by its clock through running:

C:\> net time \\[computername] /set /yes

Well, that’s all well and good, but Hal was working with NTP, which offers many more time sources than, well, our local host or domain controller. How can we pull time from an NTP server in Windows? First, please note that officially, with built-in capabilities, Windows relies on the Simple Network Time Protocol (SNTP), a stripped down NTP implementation that is often used in applications where time accuracy isn’t as vital. That’s why you’ll see sntp throughout our commands below.

Let’s first look to see which NTP server our Windows machine is configured to use:

C:\> net time /querysntp The current SNTP value is: time.windows.com,0x1 The command completed successfully.

The time.windows.com server is the default. Do you trust Microsoft to accurately track the time on their server? After missing the release date of Windows Vista by… uh… something like 3 years, maybe we shouldn’t (darnit… I couldn’t resist pouring salt in those old wounds.) We can change the NTP server we’re configured to use as follows:

C:\> net time /setsntp:pool.ntp.org The command completed successfully.
Note that you can put a whole list of NTP servers there by following the colon with a space-separated list of NTP servers enclosed in double quotes, resulting in something like /setsntp:”pool.ntp.org ntp.colby.edu tick.gatech.edu”

OK… now, once we’ve set ourselves up to use an NTP server, let’s try to check the time:

C:\> net time Could not locate a time-server.

More help is available by typing NET HELPMSG 3912.

What? Why doesn’t “net time” work here? Note that the HELPMSG is not at all helpful, as it explains that we could not locate a time-server, which we already saw in the error message itself. Gee, Microsoft… thanks for nothing.

It turns out that we can’t pull time via “net time” using NTP (really SNTP) unless we’re a domain controller with the Flexible Single Master Operation (FSMO) role. Lovely. But, what if we’re just a lowly client, maybe not even in a domain? Can we sync with an NTP server using only built in tools? As Bob the Builder and Barack Obama might say: Yes, we can!

We can rely on that happy old stand-by for configuring time on Windows boxen, the intuitively named w32tm command. And in homage to Heinz Doofenshmirtz, by “intuitively named”, I mean, of course, “completely non-intuitively named.”

To sync with an NTP server, we start by configuring our Windows box (whether it’s a client or server version of Windows) to allow us to pull time manually from an NTP server we’d like to use (note that you have to specify this with w32tm even if you’ve already run “net time” with the /setsntp option):

C:\> w32tm /config /syncfromflags:manual /manualpeerlist:pool.ntp.org The command completed successfully.
Now that we’re configured, just about ready to sync times, let’s do a quick time check before the sync:

C:\> time /t 06:47 AM

Oh… that looks dreadfully wrong. I think my clock is fast. Let’s cause Windows to read our new configuration now:

C:\> w32tm /config /update The command completed successfully.
And finally, let’s resync:

c:\> w32tm /resync Sending resync command to local computer The command completed successfully.

Boom! The clock in my tool tray is now synchronized! Let’s double check that we have the new and proper time:

C:\> time /t 06:13 AM

We’ve now synced with a remote NTP server at the command line. Yummy.

Also, there are a lot of other fascinating time-related options available with w32tm. For instance, to display your timezone, you could run:

c:\> w32tm /tz Time zone: Current:TIME_ZONE_ID_STANDARD Bias: 300min (UTC=LocalTime+Bias) [Standard Name:"Eastern Standard Time" Bias:0min Date:(M:11 D:1 DoW:0)] [Daylight Name:"Eastern Daylight Time" Bias:-60min Date:(M:3 D:2 DoW:0)]

We can even convert an NT system time (the number of 100 nanosecond intervals that have elapsed since Jan 1st, 1601, the date DOS was first written by monks) to a human readable date and time:

c:\> w32tm /ntte 128904420852883740 149194 22:21:25.2883740 - 6/25/2009 5:21:25 PM

This is especially useful if you have a Vista box (my condolences) and are researching the installation of patches using “wmic qfe”. In XP Pro and Windows 7, “wmic qfe list full” returns a list of patches with an “InstalledOn” field in a human readable form. But, in Vista, this date is now a hexadecimal number of an NT system time. Convert that hex number into decimal using you favorite calculator, and then fire up w32tm /ntte to convert it into a date you can read. Also, the “w32tm /ntpte” command converts NTP times (seconds from Jan 1st, 1900, the date that Nicola Tesla implemented the first Unix kernel) into a human readable format.

But, wait, there’s more! Hal showed how to add a timestamp into the output of your commands, useful in forensics. I sometimes will put a date and time into my command prompt itself, so that I can then copy and paste my session to use as evidence:

C:\> prompt $D$T$S$P$G Sat 10/31/2009 6:15:29.97 C:\>

So, there’s a bunch of time-related stuff. Have you got any other fun time kickers, Tim?

Tim clocks in:

Let’s take a look at the cmdlets available that use the Time or Date nouns.

PS C:\> Get-Command -Noun date CommandType Name Definition ----------- ---- ---------- Cmdlet Get-Date Get-Date [[-Date] ... Cmdlet Set-Date Set-Date [-Date] ...

PS C:\> Get-Command -Noun time [nothing]

Unfortunately, there aren’t any built-in cmdlets for NTP. So to answer Ed’s question, “No, no interesting kickers this week.” But we can still take a look at the Get-Date cmdlet and its properties.

PS C:\> Get-Date | fl

DisplayHint : DateTime DateTime : Sunday, November 01, 2009 10:17:23 PM Date : 11/1/2009 12:00:00 AM Day : 1 DayOfWeek : Sunday DayOfYear : 305 Hour : 22 Kind : Local Millisecond : 206 Minute : 17 Month : 11 Second : 23 Ticks : 633927106432066338 TimeOfDay : 22:17:23.2066338 Year : 2009

The object has a number of methods available, and they can be seen by piping the object into Get-Member. The results aren’t shown here since it is a pretty long list. Since the reason for this episode is Daylight Savings Time we’ll take a quick peek at the relevant method.

PS C:\> (Get-Date).IsDaylightSavingTime() False

Ok, that wasn’t spectacular, but let’s carry on and recreate Ed’s command prompt with the date.

PS C:\> function prompt {"PS $(Get-Date) $(Get-Location)>"} PS 11/01/2009 22:43:53 C:\>

Since the prompt is defined by overloading the prompt function there are endless possibilities for setting a custom prompt.

That’s all folks, hope to see you next time.

admin | Nov 03, 2009 | Comments 0

The Times (OK, Dates) They Are a Changing

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal finds an interesting topic:

Recently Rich Shepard, one of my colleagues on the Portland Linux User Group mailing list, posted an interesting problem. He had a data set with pipe-delimited records like:

1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|7/6/1993|0

All he wanted to do was convert the date column in the 10th field to YYYY-MM-DD format so that the file could be more easily imported into a relational database. He was curious if there was a simple command-line he could use to accomplish this.

To me, this seemed like a task that was tailor made for awk (apparently Joe Pruett agreed, since his solution on the mailing list was essentially identical to the one I’m presenting here). While awk normally splits fields on whitespace, we can use the “-F” option to specify an alternate delimiter. Once we’ve got the fields split up, we can work a little magic with the built-in split() and sprintf() operators in awk:

$ awk -F'|' '{split($10,f,"/"); $10=sprintf("%d-%02d-%02d", f[3], f[1], f[2]); print}' data 1993-1 Water Quality WVR Yamhill, City of Yamhill Hamlin Holt Npv NPDES-Waste Discharge Limits 1993-07-06 0 ...

The split() function in the example breaks up field #10 on “/” characters and puts the results into the array named “f”. Actually the last argument to split() can be a full-on egrep-style regular expression delimited with “/…/”. But since we’re just splitting on literal slash characters, “/” is a lot easier to type than “/\//”.

Once we have the year, month, and day split into an array, we then replace the contents of the 10th field with the output of the sprintf() routine. This puts our data in the desired format. The final “print” statement outputs all of the fields from the original line, including our reformatted field.

Now you’ll notice that the output is space-delimited rather than pipe-delimited. That’s because awk’s default “output field separator” (OFS for short) is space. You can actually change this by changing the value of the OFS variable. The trick is you need to set variables like this in a “BEGIN” block at the front of your awk code so that the new value is set before you begin processing your input file:

$ awk -F'|' 'BEGIN { OFS="|" } {split($10,f,"/"); $10=sprintf("%d-%02d-%02d", f[3], f[1], f[2]); print}' data 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-07-06|0 ...

Of course we could set OFS to anything. For example, we could set it to comma to produce CSV files (though there are possible quoting issues if your data contains commas). There are other variables we can set to control awk’s splitting behavior. For instance, the “-F” option is equivalent to setting the FS (“field separator”) variable. Similarly, there are the RS (“record separator”) and ORS (“output record separator”) variables, which are normally set to newline since awk operates on a line-at-a-time basis.

Anyway, if your task is chopping up data and dumping it into a different format, awk is always one good tool to reach for. I could have solved this a bit more tersely using Perl, but that would be breaking the rules for this blog. For those of you who are thinking that even my awk code is breaking the “no scripting languages” rule, it is possible to do this with cut instead of awk or sed, but the result is pretty nasty:

$ IFS='|' $ while read -a F; do printf -v d "%d-%02d-%02d" \ `echo ${F[9]} | cut -d/ -f3` \ `echo ${F[9]} | cut -d/ -f1` \ `echo ${F[9]} | cut -d/ -f2`; F[9]=$d; echo "${F[*]}"; done < data 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-07-06|0 ...

“read -a F” splits each line using the delimiter we specified in the IFS variable and assigns the fields to elements of the array named F. Notice, however, that bash indexes its arrays starting at zero (like C programs) while awk starts with one. So the date we’re reformatting is in F[9], not F[10].

The real difficulty here is that cut doesn’t let us reorder multiple fields in a single command, so we’re forced to do three instances of the “echo … | cut …” pipeline to get the date fields in the order we want. Another minor annoyance is that “printf -v …” doesn’t let us assign directly to array variables, so I have to use $d as a temporary variable.

It’s also worth pointing out that the double quotes in the last echo statement in the loop are significant. If I just wrote “echo ${F[*]}” without the double quotes, then I’d get space-separated output. Using the double quotes causes the output to be delimited with the first character of $IFS (similar to setting OFS in awk).

So there you go: an awk solution and a nasty shell-only version. Somehow I think that Tim’s Windows solution is going to look even uglier though…

Tim brings the ugly:

First off, the date format of our sample wasn’t specified, so I will assume the sample date is July 6th, 1993. My apologies to military and European followers who think the date should be June 7th, 1993.

Linux may have all sorts of different “cool” commands to use, but in the windows world we use the FOR loop…for everything.

We use our FOR loop to split the fields using the “|” and “/” characters as a delimiters. Then all we need to do is rearrange the date parts and put it all back together.

C:\> for /F "tokens=1-14 delims=|/" %a in (c:\file.txt) do @echo %a^|%b^|%c^|%d^|%e^|%f^|%g^|%h^|%i^|%l-%j-%k^|%m 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-7-6|0

Regular readers will remember the FOR loop represents the tokens by using sequential letters of alphabet. We have chosen %a to represent the first token, so %b will represent the second token, %j the 10th (month), %k the 11th (day), and %l the 12th (year). We recreate the original format by adding the “|” and “-” characters between the rearranged tokens. The problem is, if there is a “/” character in any of the text fields our results will be messed up. If we change “Water Quality” to be “Water Quality/Temp” we get these results.

C:\> for /F "tokens=1-14 delims=|/" %a in (c:\file.txt) do @echo %a^|%b^|%c^|%d^|%e^|%f^|%g^|%h^|%i^|%l-%j-%k^|%m 1993-1|Water Quality|Temp|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| 6-NPDES-Waste Discharge Limits-7|1993

We need a more robust solution that will only use the “/” character to split the date, but not the rest of the string. How do we do that? Well, if one FOR loop is good, then two must be better.

C:\> for /F "tokens=1-12 delims=|" %a in (c:\file.txt) do @for /F "tokens=1-3 delims=/" %x in ('echo %j') do @echo %a^|%b^|%c^|%d^|%e^|%f^|%g^|%h^|%i^|%z-%x-%y^|%k 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-7-6|0
The first FOR loop is used to split the string using the “|” character as the delimiter. The second FOR loop is used to split only the date field using the “/” character as a delimiter. The variables can be a little confusing so let’s take a deeper look in to the second FOR loop.

..for /F "tokens=1-3 delims=/" %x in ('echo %j') do...

This FOR loop operates on the output of “echo %j”, which is the entire date field. Using the delims option we slice the date field using the “/” character as our delimiter. The iterator in this loop is %x and it will contain the first token (month). The second and third tokens are represented by %y (day) and %z (year). Finally, we glue it all back together in the order we like using the variables created by both FOR loops.

Some of you detail oriented folks may have noticed that I neglected one point, the month and day need a leading zero. I ignored this point because this tiny change makes things really ugly. We have to use our old friend “delayed environment variable expansion” which you can read about in episodes #48, #12, and #46. Since it has been covered so many times I’ll skip some of the details for sake of brevity (ironic I know). Here is our final result:

C:\> cmd.exe /v:on /c "for /F "tokens=1-12 delims=^|" %a in (c:\file.txt) do @for /F "tokens=1-3 delims=/" %x in ('echo %j') do @set month=0%x& @set day=0%y& @echo %a^|%b^|%c^|%d^|%e^|%f^|%g^|%h^|%i^|%z-!month:~-2!-!day:~-2!^|%k" 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-07-06|0

That is a big mess and it may be difficult to see where and how the leading zero was added. Using the delayed variable expansion we set the month variable, with a leading zero, like this:

... set month=0%x& ...!month:~-2!....

The variable %x could contain 7 (July) or 11 (November). We set the variable, month, equal to the concatenation of zero and %x. The month variable would contain 07 (July) or 011 (November). Notice when the month variable is set there is no space between the variable (%x) and the “&” character. If we did leave a space then our month variable would contain a trailing space which would later have to be removed. When we echo the month variable we only want the two rightmost characters so July is displayed as 07 and November as 11. The same process is used for the day of the month.

Powershell:

Powershell gives us the ability to use regular expressions which makes everything much easier. We can reformat any date in our file using this command:

PS C:\> Get-Content file.txt | ForEach-Object { $_ -replace '(\d{1,2})/(\d{1,2})/(\d{4})','$3-$1-$2' } 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-7-6|0

The Get-Content commandlet (alias gc) returns each line of the file given. Using the ForEach-Object (alias %) we operate on each line of the file. The “current pipeline object”, represented as $_, contains the content of the current line in the file (in our example we only have one line in our file).

Our regular expression search and replace finds the month/day/year and rearranges it as year-month-day. Again we have the problem of adding that pesky leading zero so we need to use a slightly different command.

PS C:\> gc file.txt | % { $_ -replace '(\d{1,2})/(\d{1,2})/(\d{4})', '$3-0$1-0$2' } | % { $_ -replace '(\d{4}-)\d?(\d{2}-)\d?(\d{2})','$1$2$3'} 1993-1|Water Quality|WVR|Yamhill, City of|Yamhill|Hamlin|Holt|Npv| NPDES-Waste Discharge Limits|1993-07-06|0

We use two replace commands in order to add our leading zero. The first replace command adds a leading zero and rearranges our month/day/year, resulting in year-0month-0day. The second command removes the leading zeros if they are unnecessary.

admin | Oct 13, 2009 | Comments 0

Just Sit Right Back & You’ll Hear a Tale… or a Tail…

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Ed muses whimsically:

I’m sure every self-respecting geek has contemplated the scenario. I know I think about it all the time. You’re trapped on a desert island, surrounded by nothing but coconut trees, sand, water, and 50 beautiful babes… all living in rustic harmony in your lavish hut. Oh, and you also have your laptop computer, with a power supply and Internet connection. Now, the question before the house, of course, is as follows:

When stranded on a desert island, if you could only have a single command in your operating system, what would it be and how would you use it?

Yes, it’s a dilemma that has no doubt puzzled philosophers for ages. I’d like to weigh in on it, and see Hal’s thoughts on the matter as well.

For my Windows usage, I’d have to go with WMIC, the Windows Management Instrumentation Command-Line tool. While it’s just a command, it opens up whole worlds to us for interacting with our Windows boxen. Built-into Windows XP Pro and later, WMIC can be used to query information from machines and update it using a syntax known as the WMIC Query Language (WQL), which I described in an article a while back.

WMIC can be used as a replacement for numerous other commands, and in many instances it provides even more information than the commands it subsumes. For instance, you could supplant the Service Controller command (sc) that we discussed in Episode #57 with:

C:\> wmic service list full

Or, you can replace the tasklist command with:

C:\> wmic process list full

The taskkill command functionality can be mimicked with:

C:\> wmic process where processid="[pid]" delete

You can get lists of users, including many settings for their account and their SIDs with:

C:\> wmic useraccount list full

Those are the things I most often use WMIC for: interacting with services, processes, and user accounts based on variations of those commands. I’ve written a lot about WMIC in the past, but I’ve come up with some new uses for it that I’d like to talk about here. And, getting back to our little desert island fantasy… I mean… scenario, let’s talk about some additional WMIC use cases.

Suppose, on this desert island, you wanted to see if a given Windows machine was real or virtual. Perhaps you had hacked into another box on the island or you had this question about your own system. WMIC can provide insight into the answer, especially if VMware is in use:

C:\> wmic bios list full | find /i "vmware" SerialNumber=VMware-00 aa bb cc dd ee ff aa-bb cc dd ee ff 00 aa bb

VMware detection, in a single command! I’m sure the babes will like that one. Here, I’m querying the bios of my Windows machine, looking for the string “VMware” in a case-insensitive fashion (/i). If you see output, you are running inside a VMware guest machine. Also, you’ll get the serial number of that VMware install, which might be useful to you.

Perhaps, with all that spare time on your island paradise, you will start to contemplate the fine-grained inner workings of your Windows box, thinking about the order that various drivers are loaded. Wanna see that info? Use this:

C:\> wmic loadorder list full

On a desert island, I’m sure you’ll need to know a lot of details about your hard drive, including the number of heads, cylinders, and sectors (so you can make a new hard drive from coconut shells when your existing one fails, of course). To get that information, run:

C:\> wmic diskdrive list full

At some point, you may need to write up a little command-line script that checks the current screen resolution on your default monitor. There must be a distinct need on desert islands for pulling this information (perhaps just to impress the babes), which can be obtained with:

C:\> wmic desktopmonitor where name="Default Monitor" get screenheight,screenwidth ScreenHeight ScreenWidth 665 1077

Now, suppose you are conducting a detailed forensics investigation to determine who among your cadre of babes stole the coconut cream pie. The answer might lie in the creation, modification, or last accessed time of a given directory on your hard drive. You can get that information by running:

C:\> wmic fsdir where (name="c:\\tmp") get installdate,lastaccessed,lastmodified InstallDate LastAccessed LastModified

20090913044801.904300-420 20090914051243.852518-420 20090913073338.075232-420

Note that the path to the directory in this one must use \\ in place of each backslash. The first backslash is an escape, and the second is the real backslash. You have to do this for any where clauses of wmic that have a backslash in them. Also, note that fsdir works only for directories, not files. Still, that should help you crack the case of the missing coconut cream pie!

There are thousands of other uses for WMIC, which can be explored by simply running “wmic /?”. As you can see, it is an ideal tool for an intrepid geek in a tropic island nest.

No phone! No lights! No motor cars!
Not a single luxury…
Like Robinson Caruso…
Except for WMIC. :)

Hal’s been on the island far too long:

When Ed proposed this question I thought it was kind of unfair for me to get to choose a command plus have all the functionality of the Unix shell. And that got me thinking, just how much could I accomplish using the shell itself with no other external commands? This is not as idle a question as it might first appear: there have been times when I’ve had to recover hosed systems without much more than the built-in functionality in my shell.

First let’s inventory our resources. Built into the bash shell we have cd for navigating the directory tree, echo and printf for outputting data, read for reading in data, and a few miscellaneous commands like kill, umask, and ulimit. We also have several different kinds of loops and conditional statements, plus the test operator for doing different kinds of comparisons. This actually turns out to be a lot of functionality.

Starting off simply, we can create a simple ls command with the echo built-in:

$ cd /usr/local $ echo * bin cronjobs depot etc games include lib lib64 libexec lost+found man sbin share src

But that output is kind of ugly, and would be hard to read if the directory contained more items. So we could make pretty output with an ugly loop:

$ i=0; for f in *; do printf '%-20s' $f; (( ++i % 4 )) || echo; done; \ (( $i % 4 )) && echo bin cronjobs depot etc games include lib lib64 libexec lost+found man sbin share src

I’m using printf to output the data in columns (though you’ll note that my columns sort left to right rather than up and down like the normal ls command), and a counter variable $i to output a newline after the fourth column.

Emulating the cat command is straightforward too:

$ while read l; do echo $l; done file" to zero-out a file). There's also no way to do the ln command in the shell, nor to emulate commands like chown, chmod, and touch that update the meta-data associated with a file.

However, since bash has a pattern matching operator, we can emulate grep very easily:

$ while read l; do [[ $l =~ ^127 ]] && echo $l; done 127.0.0.1 localhost.localdomain localhost
In a similar vein, we can also count lines ala “wc -l”:

$ i=0; while read l; do ((i++)); done 4

While our cat emulator works fine for small files, what if we had a longer file and wanted something like more or less that would show us one screenful at a time:

$ i=0; \ while read -u 3 l; do echo $l; ((++i % 23)) || read -p 'More: '; done 3 root:x:0:0:root:/root:/bin/bash [... 21 lines not shown ...] smmsp:x:51:51::/var/spool/mqueue:/sbin/nologin More:

After every 23 lines, I use the read command to display the “More: ” prompt and wait for the user to hit newline. Since I’m going to be reading the user’s input on the standard input, I have to read the file the user wants to view on a different file descriptor. At the end of the loop I’m associating the /etc/passwd file with file descriptor 3, and at the top of the loop I use “read -u 3″ to read my input from this file descriptor. Thank you bash, and your amazingly flexible output redirection routines.

Since we have for loops, creating our own version of the head command is also easy:

$ for ((i=0; $i<10; i++)); do read l; echo $l; done root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt mail:x:8:12:mail:/var/spool/mail:/sbin/nologin news:x:9:13:news:/etc/news:

If I have the head command, I suppose it’s a moral imperative that I also produce something like tail:

$ i=0; while read l; do a[$i]=$l; i=$(( ($i+1)%10 )); done for ((j=0; $j<10; j++)); do echo ${a[$(( ($j+$i)%10 ))]}; done webalizer:x:67:67:Webalizer:/var/www/usage:/sbin/nologin pcap:x:77:77::/var/arpwatch:/sbin/nologin hsqldb:x:96:96::/var/lib/hsqldb:/sbin/nologin xfs:x:43:43:X Font Server:/etc/X11/fs:/sbin/nologin gdm:x:42:42::/var/gdm:/sbin/nologin sabayon:x:86:86:Sabayon user:/home/sabayon:/sbin/nologin radiusd:x:95:95:radiusd user:/:/bin/false mailman:x:41:41:GNU Mailing List Manager:/usr/lib/mailman:/sbin/nologin tomcat:x:91:91:Tomcat:/usr/share/tomcat5:/bin/sh avahi-autoipd:x:100:103:avahi-autoipd:/var/lib/avahi-autoipd:/sbin/nologin

In the first loop I’m using an array as a circular buffer to hold the last 10 lines read. After the first loop exhausts the file, I use a second loop to output the lines stored in the array.

The idea of reading the contents of a file into an array suggested this nasty hack to emulate the sort command:

$ n=0; while read l; do a[$n]=$l; ((n++)); done n=$(($n-1)); \ for ((i=0; $i<$n; i++)); do s=$i; for ((j=$((i+1)); $j<=$n; j++)); do [[ ${a[$s]} < ${a[$j]} ]] || s=$j; done; t=${a[$i]}; a[$i]=${a[$s]}; a[$s]=$t; done; \ for ((i=0; $i<=$n; i++)); do echo ${a[$i]}; done 1 1 10 10 2 3 4 5 6 7 8 9

Yep, the middle, nested loops are actually a selection sort implemented in the shell. You’ll notice that the sort here is an alphabetic sort. We could produce a numeric sort using “-lt” instead of “<” inside the “[[ ... ]]” clause in the innermost loop.

You’ll also notice that I put some duplicate values in my test input file. Hey, if you’re going to do “sort” you’ve got to also do “uniq”. Here’s a numeric sort plus some mods to the final loop to emulate uniq:

$ n=0; while read l; do a[$n]=$l; ((n++)); done n=$(($n-1)); \ for ((i=0; $i<$n; i++)); do s=$i; for ((j=$((i+1)); $j<=$n; j++)); do [[ ${a[$s]} -lt ${a[$j]} ]] || s=$j; done; t=${a[$i]}; a[$i]=${a[$s]}; a[$s]=$t; done; \ for ((i=0; $i<=$n; i++)); do [[ "X$l" == "X${a[$i]}" ]] || echo ${a[$i]}; l=${a[$i]}; done 1 2 3 4 5 6 7 8 9 10

With the help of the IFS variable, we can do something similar to the cut command:

$ IFS=":"; \ while read uname x uid gid gecos home shell; do echo $uname $uid; done root 0 bin 1 daemon 2 ...

And since bash has a substitution operator, I can even emulate “sed s/…/…/”:

$ while read l; do echo ${l//root/toor}; done toor:x:0:0:toor:/toor:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin ...

I couldn’t resist exploiting /proc on my Linux box to generate a simple ps listing:

$ printf "%-10s %5s %5s %s\n" UID PID PPID CMD; \ for d in /proc/[0-9]*; do cmd=$(cat $d/cmdline | tr \\000 ' '); while read label value rest; do case $label in Name:) name=$value;; Pid:) pid=$value;; PPid:) ppid=$value;; Uid:) uid=$value;; esac; done <$d/status; [[ -z "$cmd" ]] && cmd=$name; printf "%-10s %5s %5s %s\n" $uid $pid $ppid "$cmd"; done UID PID PPID CMD 0 1 0 init [3] 0 10 1 watchdog/2 0 10994 87 kjournald 0 11 1 migration/3 0 11058 1 /usr/lib/vmware/bin/vmware-vmx -...

This is obviously skirting pretty close to our “no scripting” rule, but I actually was able to type this in on the command line. I suspect that there may be information available under /proc that would also enable me to emulate some functionality of other commands like netstat and ifconfig, and possibly even df, but this episode is already getting too long.

Before I finish, however, I wanted to show one more example of how we could create our own simple find command. This one definitely wanders far into scripting territory, since it involves creating a small recursive function to traverse directories:

$ function traverse { cd $1; for i in .[^.]* *; do $(filetest $i) && echo "$1/$i"; [[ -d $i && -r $i && ! -h $i ]] && (traverse "$1/$i"); done; } $ function filetest { [[ -d $1 ]]; } $ traverse /etc traverse /etc /etc/.java /etc/.java/.systemPrefs /etc/acpi /etc/acpi/ac.d /etc/acpi/battery.d /etc/acpi/events /etc/acpi/resume.d /etc/acpi/start.d /etc/acpi/suspend.d /etc/alternatives ...

Specify a directory and the traverse function will walk the entire directory tree, calling the filetest function you define on each object it finds. If the filetest function resolves to true, then traverse will echo the pathname of the object it called filetest on. In the example above, filetest is true if the object is a directory, so our example is similar to “find /etc -type d”.

admin | Sep 22, 2009 | Comments 1

Fishing for Network Configs

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Ed kicks it off:

Man, we’ve covered a lot of topics in our 54 episodes prior to this one. But, in our rush to get you the latest chocolate-covered command-line fu, occasionally we’ve missed some fundamentals. People write in with questions (which we love) about such items, inspiring a new episode. Back in May, we received a great question from Johnny C:

> I have a suggestion for command line kungfu.
> I need to be able to change my IP Address back and forth from DHCP
> where everything is dynamic to a dedicated IP address.
> I’ve worked with this for a while and my problems have been not able
> to update DNS on Windows

Ah… good one, sir! Let’s face it: the built-in Windows GUI associated with network configuration changes is horrible… forcing numerous clicks through various screens to make even small tweaks. At least we don’t have to live through the dreaded reboots of the Windows 95 era just to change IP addresses anymore.

On Windows, for manipulating network configs at the command line, netsh rocks, and it can do what you want, Johnny C, and much more. In fact, when I’ve got a lazy summer afternoon with nothing better to do, I fire up netsh (or the equally fun and interesting wmic command) and just explore, sometimes for hours on end. The netsh command (like wmic) can run in two modes: either as a little command interpreter of itself (by typing netsh and hitting Enter) lending itselve to exploration, or as a single shot command of netsh followed by various options.

To get a glimpse of the capabilities of netsh, run the following:

C:\> netsh netsh> ?

The following commands are available:

Commands in this context: .. - Goes up one context level. ? - Displays a list of commands. abort - Discards changes made while in offline mode. add - Adds a configuration entry to a list of entries. advfirewall - Changes to the `netsh advfirewall' context. alias - Adds an alias. bridge - Changes to the `netsh bridge' context. bye - Exits the program. commit - Commits changes made while in offline mode. delete - Deletes a configuration entry from a list of entries. dhcpclient - Changes to the `netsh dhcpclient' context. dump - Displays a configuration script. exec - Runs a script file. exit - Exits the program. firewall - Changes to the `netsh firewall' context. help - Displays a list of commands. http - Changes to the `netsh http' context. interface - Changes to the `netsh interface' context. ipsec - Changes to the `netsh ipsec' context. lan - Changes to the `netsh lan' context. nap - Changes to the `netsh nap' context. netio - Changes to the `netsh netio' context. offline - Sets the current mode to offline. online - Sets the current mode to online. p2p - Changes to the `netsh p2p' context. popd - Pops a context from the stack. pushd - Pushes current context on stack. quit - Exits the program. ras - Changes to the `netsh ras' context. rpc - Changes to the `netsh rpc' context. set - Updates configuration settings. show - Displays information. unalias - Deletes an alias. winhttp - Changes to the `netsh winhttp' context. winsock - Changes to the `netsh winsock' context. wlan - Changes to the `netsh wlan' context.

Nice! Lots of very useful stuff, including “interface” and “firewall” (the latter of which we discussed in Episode #30). There’s also some really nifty settings for ipsec (on 2003 and later) and wlan (on Vista and later) contexts. To change to an individual context, just type its name (such as “interface”) and then type ? at the netsh> prompt to get more info about it. You can then navigate down by entering follow-up commands and contexts, and then pop back up to earlier contexts entering a command of dot-dot (“..”). I wish there was a “back” command instead of .., but I can cope. There’s even a pushd and popd command for netsh contexts, rather similar to the pushd and popd for directories we discussed in Episode #52.

One of my most common uses of netsh is to change IP address settings of the machine. In the spirit of the cliche “Give a man a fish and feed him for a day… teach him to fish and feed him for life”, let me show you how you can fish around inside of netsh.

We first invoke netsh and then move to the interface context:

C:\> netsh netsh> interface netsh interface> ?

Here, you can see options for various elements we can configure on the machine. Of particular interest to us now is ip (on XP and 2003) or ipv4 (on Vista and later). Happily, you can just type “ip” on Vista, and it will take you to the ipv4 context, so our netsh commands for changing addresses and such are compatible between various versions of our beloved Windows operating system.

netsh interface> ip netsh interface ip> set ?

Now, we can get a sense of the various items we can set, including addresses, dns, and wins. But, wouldn’t it be nice if Windows would give us examples of how to set each? Well, ask and ye shall receive:

netsh interface ip> set address ? Usage: set address [name=] [[source=]dhcp|static] [[address=][/] [[mask=] ] [[gateway=]|none [gwmetric=]] [[type=]unicast|anycast] [[subinterface=] ] [[store=]active|persistent]
Examples:

set address name="Local Area Connection" source=dhcp set address "Local Area connection" static 10.0.0.9 255.0.0.0 10.0.0.1 1

If you’d like to get a list of all interfaces available on the machine, you could run (from the normal C:\> prompt, not within netsh):

C:\> netsh interface show interface

I know… it looks like it is redundantly repeating itself twice back to back, and it is. But, that’s the command. Now, we know how to refer to our network interfaces for manipulating them.

Then, to set an IP address, we could just run the command:

C:\> netsh interface ip set address name="Local Area Connection" static 10.10.10.10 255.255.255.0 10.10.10.1 1

This will set our IP address to 10.10.10.10, with a netmask of 255.255.255.0, a default gateway of 10.10.10.1, and a routing metric (number of hops to that gateway) of 1.

For DHCP, we simply run:

C:\> netsh interface ip set address name="Local Area Connection" source=dhcp

OK…. now to answer Johnny C’s specific question, setting our primary DNS server:

C:\> netsh interface ip set dnsserver name="Local Area Connection" static 10.10.10.85 primary

And, if you’d rather get that info from DHCP, you could use:

C:\> netsh interface ip set dnsserver name="Local Area Connection" source=dhcp
I frequently find myself changing between my laboratory network and my production network, which have completely different IP addressing schemes. To help make a quick switch between them, I don’t use those one of those goofy network configurator GUIs, because, well, they are kind of tawdry. Instead, I’ve created two simple scripts that I keep on my desktop: test.bat and prod.bat. Each one contains two netsh commands. The first command sets my IP address, netmask, and default gatewy for either prod or test, and the second command sets my DNS server. When I want to invoke them, I simply run them with admin privs (based on either being logged in as admin, or right clicking and selecting “run as administrator”).

Hal kicks it old school:

Where the Windows way is to have one big command that does everything, remember the Unix design religion is to have a bunch of small commands that do simple things and then combine them to produce the effect you want. It doesn’t help that Unix systems have been networked devices since the earliest days of the Internet– we’ve got multiple generations of command interfaces to deal with. But let me try to hit the high points.

Suppose our system is normally configured to use DHCP but we want to manually move it onto a new network with a static address assignment. Step one is to change your IP address with the ifconfig command:

# ifconfig eth0 10.10.10.1 netmask 255.255.255.0 # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:0C:29:18:C3:0D inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe18:c30d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:55 errors:0 dropped:0 overruns:0 frame:0 TX packets:158 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7542 (7.3 KiB) TX bytes:32712 (31.9 KiB) Interrupt:18 Base address:0x2024
As you can see from the example above, you can also use ifconfig to display information about an interface’s configuration (“ifconfig -a” will display the configuration information for all interfaces on the system).

However, changing the IP address with ifconfig doesn’t have any impact on your routing table. You’ll probably need to add a default route when you change the IP address:

# route add default gw 10.10.10.254 # netstat -rn Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 0.0.0.0 10.10.10.254 0.0.0.0 UG 0 0 0 eth0

I should note that as you move from on Unix distribution to another, the route command syntax tends to change in minor ways, rendering a command that is syntactically correct on one system completely useless on another. The above example is for Linux. Oh and by the way, if your DHCP configuration has created a default route for the network you used to be on, you can remove it with “route del default gw “.

The last thing you have to do is update your list of local DNS servers. This list is configured in /etc/resolv.conf. Most DHCP clients will simply overwrite the contents of this file with the name servers and local domain they learn from their DHCP servers, but you can also edit this file directly. A sample file might look like:

nameserver 10.10.10.100 search somedomain.com

Replace “somedomain.com” with the default domain you want the host to use for looking up unqualified host names. You can have multiple “nameserver” lines in your file for redundancy. However, I warn you that the timeout on the first lookup is long enough that your users will pick up the phone and call you to tell you the “network is down” before the system fails over to the next name server in the list.

The combination of ifconfig, route, and editing your resolv.conf file should be sufficient to get you manually moved onto a new network. The more interesting question is how to you revert back to using DHCP to configure your network interface? Assuming your machine is configured by default to use DHCP, the easiest thing is to just shut down and then reactivate your network interface. Of course the process for doing this is completely different for each Unix system you encounter. On most Linux systems, however, the following will work:

# ifdown eth0 # ifup eth0

Determining IP information for eth0... done. # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:0C:29:18:C3:0D inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fe18:c30d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:56 errors:0 dropped:0 overruns:0 frame:0 TX packets:228 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7884 (7.6 KiB) TX bytes:44273 (43.2 KiB) Interrupt:18 Base address:0x2024

By the way if you’re looking for your static network interface configuration information, you’ll find it in /etc/sysconfig/network-scripts/ifcfg-eth0 on Red Hat systems (including CentOS and Fedora) and in /etc/network/interfaces on Debian systems. This is the place where you can set the default interface configuration parameters to be used when booting. Be aware, however, that on modern Ubuntu systems network configuration is under the control of NetworkManager by default– a GUI-based network configuration tool very reminiscent of the Windows network configuration GUI. Helpful for people coming over from the Windows environment I guess, but kind of a pain for us old farts who are used to configuring network interfaces with vi.

admin | Aug 11, 2009 | Comments 0

chmod Squad

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal says:

I was doing some work for a customer recently that involved a lot of manipulating file system permissions and ownerships and I realized that we’ve not yet had a Command Line Kung Fu Episode on this subject. That’s just crazy! So let’s address this shortcoming right here and now.

You’re probably familiar with the chown command for changing the owner (and group owner, if desired) of a file or group of files. Add the “-R” (recursive) option, and you can set the ownerships on an entire directory structure:

# chown -R hal:users /home/hal

These days you typically have to be root to use the chown command. In the olden days of Unix, it was common to allow any user to use chown and let them “give away” their own files to other users. But this caused problems for systems that used disk quotas because it was a loophole that allowed malicious users to consume other users’ quotas. And in the really old days the chown command wouldn’t strip execute bits when files were chowned, thus allowing you to create set-UID binaries belonging to other users– obviously a huge security problem.

The problem with having to chown everything as root, however, is that all too often I see administrators making a mistake like this:

# cd /home/hal # chown -R hal:users .* # *NEVER* DO THIS! DANGER!

Why is the above chown command so dangerous? The problem is that the glob “.*” matches the “..” link that points back to the parent directory of the current directory. So in the example above we’re going to end up making the entire /home file system owned by “hal”. This is actually incredibly difficult to recover from, short of restoring from a backup, because it’s not safe to assume that every file in a user’s home directory is going to be owned by that user.

Anyway, the safe way to chown a user’s “dot-files” and directories is:

# chown -R hal:users .[^.]* # SAFE

The “[^.]” means “match any character EXCEPT period”, thus protecting you from matching the “..” link. Of course it would also skip files and directories named things like “..example”, but you wouldn’t expect to find these in a typical file system.

Having set the ownerships on a set of files, you can also control the access rights or permissions on those files. There are three categories of permissions– the permissions for the primary owner of the file (“user” permissions), the group owner of the file (“group” permissions), and “everybody else” (“other” in Unix parlance). For each group you can allow “read” access (view the contents of a file or get a listing of a directory), “write” (modify the contents of a file or add, remove, and/or rename files in a directory), and/or “execute” privileges (execute a file as a program or access files in a directory). In “absolute” mode with the chmod command, we express these permissions as a vector of octal digits: “read” is 4, “write” is 2, and “execute” is 1. Here are some common examples:

$ chmod 755 /home/hal # rwx for me, r+x for group and other $ chmod 666 insecure # rw for everybody, "world writable" $ chmod 700 private.dir # only I have access here

Because the default assumption for the Unix operating system is to create world-writable files and directories, we use the umask value to express which bits we want NOT to be set when new files are created. The common Unix umask default is “022″ which means “don’t set the write bits for group and other”. Some sites enforce a default umask of “077″, which requires you to explicitly use chown to all others to access your files (“discretionary access control” is the term usually bandied about here).

There’s actually an optional leading fourth digit you can use with the chmod command, which covers the “set-UID” (4), “set-GID” (2), and “sticky-bit” settings (1). Here are some examples showing the typical permission settings for some common Unix programs and directories:

# chmod 4755 /bin/su # set-UID # chmod 2755 /usr/sbin/sendmail # set-GID # chmod 1777 /tmp # "sticky"
The “sticky-bit” is another interesting piece of Unix history. Back in the very old days when computers, disks, and even memory were vastly slower than they are today, there was a significant start-up cost with loading a program into memory before execution. For commonly-used programs like ls, the total overhead was enormous. So certain executables were marked with the “sticky-bit” as a signal to the kernel so that the program image would tend to “stick around” in memory so that the program didn’t have to constantly be reloaded. Of course, in the age of shared libraries plus fast disks and memory this use has long since stopped having any value. Nowadays, the sticky bit is used as a marker on world-writable directories like /tmp and prevents anybody except the owner of the file from removing or renaming the file.

As you’re probably aware, chmod also supports “symbolic” mode for expressing changes to the permissions of files:

$ chmod go-w insecure # strip the write bits for group and other $ chmod a+x myscript # make file executable for everybody ("all") $ chmod +x myscript # same as above

The first part of the syntax is the groups to apply the permission change to: “u” for the primary user or owner of the file, “g” for the group owner, and “o” for other or everybody else. You can also use “a” to represent all groups, or just leave it off completely because “a” is the default. The next item in the symbolic description is a plus or a minus sign depending on whether you’re “adding” or “subtracting” permission bits. Finally you specify the bit(s) you want to add or subtract.

Why is symbolic mode useful? Well, I recently needed to make an entire directory structure be only accessible by its owner. You can’t really use absolute file modes for this, because while “chmod -R 600 …” would work fine for regular files, it wouldn’t do for the directories (directories must have “x” set to be usable). “chmod -R 700 …” would be fine for directories, but not appropriate for regular files. You could hack something together with find, but it’s much easier to just do:

# chmod -R go-rwx /some/directory

This strips all bits for the “group” and “other” categories, but leaves the current permissions for the owner of the file alone.

Unfortunately, I now have to turn things over to Ed. I say “unfortunately”, because I’m going to have to endure his gloating about the higher level of permissions granularity implemented in NTFS as compared to the typical Unix file system. To this my only response is, “Oh yeah? Tell me about ownerships and permissions in FAT file systems, boyo.”

Ed responds:

I thought we had an agreement to never bring up FAT. As far as I’m concerned, it never happened. NTFS has been with us since Day 1. Yeah… right.

Anyway, since Hal brought up FAT, I’ll address it briefly. There is no concept of security with FAT file systems. Every user is God and can access everything. It’s a relic from the days when DOS (and later Windows) machines were expected to be single-user systems not connected to, you know, a network or anything. The security model was to diligently implement no security model whatsoever. Mission accomplished!

But, then, there came NTFS. Ahhh… finally, a modern file system for Windows boxen. NTFS has the concept of ownership and even more fine-grained controls than our Unix brethren have. As Windows NT matured into Windows 2000 and then XP/2003, and then Vista/2008 and beyond, these fine-grained permissions and the ability to manipulate them at the command line has expanded significantly. In fact, we’re now to the point where these features are simultaneously flexible enough to use yet complex enough to be overwhelming and almost unusable. That happens a lot in the Windows world.

From the command-line, we can see the ownership of a file or directory using the dir command with the /q option:

C:\> dir /q

Nice! Easy! Cool!

Yeah, but wait. Now things get ugly. If the hostname\username is particularly long, it will be truncated in the display, which allocates only 23 characters for the hostname backslash username. None of the other output formatting options of dir (/b, /w, /d, and /x) fix this, because they either override /q (leaving off the ownership info) or keep the truncation. This problem is livable, but still kind of annoying.

To change the owner of a file or directory, a la the Linux chown command, we can use the icacls command found in 2003 SP2, Vista, 2008, and Windows 7, as follows:

C:\> icacls [filename] /setowner [username]

To change the owner in a recursive fashion, use the /t option with icacls, because, as everyone knows the word “recursive” has no letter t in it. That makes it easy to remember, right? (Actually, I think they were going after the mnemonic for “tree”, but a /s or even a /r would be more palatable). And, /c makes icacls continue despite an error with one or more files as it is processing through a /t or wildcard.

Now, look at that list of supported Windows versions for icacls… there’s an important one missing from the list. Which one? Queue the Jeopardy music and pause. Sadly, it’s our good friend, Windows XP. Unfortunately, I haven’t found a way at the command line using only built-in tools to change owner in XP. You could mount the XP system’s drive from a 2003/Vista/2008/7 box and run icacls, but that’s not exactly using built-in tools, now, is it? You can copy the 2003 SP2 version of icacls.exe to Windows XP, and it’ll work… but again, that’s not exactly using built-in tools, and I have no idea whether that violates some sort of Windows license limitation. And, I don’t want to know, so don’t send me any mail about it.

The Vista version won’t run on XP though. I’ve also found that icacls on 2003 (whether running in 2003 or… ahem… copied to XP) is quite buggy as well, often giving errors when trying to change owners. This 2003 icacls fail is a known issue documented by Microsoft, and they released a hotfix for it which is seldom installed. So, does this count as not using a built-in command? :)

To change ownership in the GUI on XP, you can apply a ridiculous process described by Microsoft here.

Now, XP does include the cacls command (not icacls), as does Win2K and all of the later versions of Windows. The cacls command lets you change permissions, the rough equivalent of the Linux chmod command. But, cacls will not let you change the owner.

The syntax for the cacls command lets us specify the file or directory name we want to change, followed by a bunch of potential options. We can grant access rights with /G [user:perm]. The perms supported at the command line are R (Read), W (Write), C (Change, sometimes referred to as “Modify” in Windows documentation), and F (Full Control). These rights are actually conglomerations of the very fine grained rights built-into NTFS, which are described here.

We can revoke these access rights with /R [user]. Note that you cannot revoke individual rights (R/W/C/F), but instead you revoke all of them at a given time for a user. Revocation is an all-or-nothing situation. The /E is used to edit the existing ACL, as opposed to the default, which replaces the ACL. We often want to use /E so that we don’t blow away any access rights already there. There is also a /D [user] option in cacls, which explicitly denies the user access to the object, again on an all or nothing basis. These deny rights override any allow rights, thankfully.

With that overview under our belts, to frame the following fu, I’d like to mimic Hal’s commands, mapping them into the Windows world to the extent we can.

We start with Hal’s first command:

$ chmod 755 /home/hal # rwx for me, r+x for group and other

In Windows, we can achieve the same thing with these three commands:

C:\> cacls [filename] /G [username]:F C:\> cacls [filename] /E /G [groupname]:R C:\> cacls [filename] /E /G Everyone:R

Note that the R here is a conglomerated Read, which includes reading and executing the file (again, those conglomerated rights are defined here). Also, in the first command of these three, we’ve not used /E, so we are blowing away all existing access rights to start out and adding full control for our username in the first command. Hal’s assigning permissions absolutely, not relative to existing permissions, so we leave off the /E. In our follow-up command, though, we edit rights (/E) building on with a couple of extra commands to match roughly what Hal has done.

Next, our sparring buddy ran:

$ chmod 666 insecure # rw for everybody, "world writable"

We can roughly mimic this with:

C:\> cacls [filename] /G [username]:F C:\> cacls [filename] /E /G Everyone:C

Now, the execute capability is baked into both Full control (F) and Change (C), so we’re not really removing execute capability here. With icacls (not cacls), we can access the fine-grained rights and make a file read-only with:

C:\> icacls [filename] /grant Everyone:RW

Remember, /G is for cacls, and /grant is for icacls. Consistency is a beautiful thing.

And then, Hal pulled this out of his ear:

$ chmod 700 private.dir # only I have access here

In our comfy Windows world, we could run:

C:\> cacls [filename] /G [username]:F

Without the /E above, this snips off everyone else’s access.

Hal then wowed us all with:

$ chmod go-w insecure # strip the write bits for group and other

This one is a bear to do at the command-line in Windows, because the /R option is all or nothing when revoking permissions. We’d need to analyze the existing ACL first, and then manipulate it with a /E to build the ACL we want. It would require a script to do this by my estimation.

Hal then popped off:

$ chmod a+x myscript # make file executable for everybody ("all")

Which we can do with:

C:\> cacls [filename] /E /G Everyone:R

Again, in cacls, R includes Read and Execute.

And, finally, Hal did:

# chmod -R go-rwx /some/directory

Which, mapped to our own personal insanity, is:

C:\> cacls [directory] /p [user]:F /t

In this one, we’ve used the /p to replace the existing ACLs for the given user name, which we are giving full control (F), in a recursive fashion (/t).

Be careful when playing with icacls and cacls, because they are a great way to very much hose up your system. The icacls command has an option to save ACLs into a file for later inspection or even restoring from:

C:\> icacls [file_or_dir] /save [aclfile]

Again, we have a /t or /c option here. The restore function is invoked with /restore. It should be noted that the aclfile holds only relative paths. Thus, if you are going to do a restore, you need to run the command in the same directory that you used for the icacls /save.

admin | Aug 04, 2009 | Comments 0

The Final Coundown – timers and loops and things

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

The Final Countdown

Ed starts:

My 40th birthday is coming up this Tuesday (the day that this episode will be posted), so I’ve been thinking about the passage of time a lot lately, with countdowns, hour glasses, and ticking clocks swirling through my thoughts. This all came to a head a couple of days ago, when I received an e-mail from a buddy of mine who runs capture the flag games. He wanted a countdown timer for his games, something to spice up players’ experience in the game. But, instead of using some lame clock or stopwatch app, he asked me to provide him some command-line kung fu to do the trick. I came up with a solution that he liked, and it includes some fun twists and turns that I thought our readers here may find useful or at least enjoyable.

I relied heavily on FOR loops in the following command:

C:\> for /l %m in (1,-1,0) do @for /l %s in (59,-1,0) do @echo %m minutes %s seconds LEFT & ping -n 2 127.0.0.1 > nul 1 minutes 59 seconds LEFT 1 minutes 58 seconds LEFT 1 minutes 57 seconds LEFT 1 minutes 56 seconds LEFT 1 minutes 55 seconds LEFT

Here, I’ve made a countdown timer that will run for two minutes (starting at minute 1, and counting back 60 seconds from 59 to 0, then going to minute zero and counting back the same way). I start off my minute variable (%m), running it from 1 down to 0 in steps of -1. If you want a 10 minute counter, replace that 1 with a 9. My second variable (%s) runs from 59 down to 0, again in steps of -1. At each iteration through the loop, I print out how much time is left. Finally, I ping myself twice, which takes about* a second (the first ping happens immediately, the second happens about* 1 second later).

That was my starting point. But, I wanted to add some flair, adding a little audio and popping up a message on the screen at the end of each minute. So, I added:

C:\> (for /l %m in (1,-1,0) do @(for /l %s in (9,-1,0) do @(echo ONLY %m minutes %s seconds LEFT & ping -n 2 127.0.0.1>nul)) & start /max cmd.exe /c "echo ^G %m MINUTE^(s^) LEFT! & ping -n 6 127.0.0.1>nul") & echo ^G^G^G^G^G^G^G^G^G

Here, as each minute expires, I’m using the start command to run a program in a separate window (which start does by default… the /b option makes start run a program in the backgroudn of the same window as we discussed in Episode #23 on job control) maximized on the screen with the /max option. The start command allows me to kick off something else while my main loop keeps running, ticking off seconds into the next minute.

The program that the start command will launch is cmd.exe, which I’m asking to execute a command with the /c flag. The command run by cmd.exe will echo a CTRL-G, which makes the system beep, and then echoes the number of minutes left. We then wait for 5 seconds (by pinging localhost 6 times). Because I started the cmd.exe with a /c option, after its command finishes (5 seconds later), the window will disappear. The location of all those parens is vitally important here, so that we’re properly grouping our commands together to notch off time.

To add even a little more flair, I added several CTRL-Gs after the timer is done to make the expiration of the full time more audible.

* OK…. yes, you are right. The countdown timer I’ve describe here isn’t super accurate, because the 1-second rule on pings is an approximation. Also, some time will be consumed by the commands gluing this all together, making this stopwatch slower than it should be. Also, if the system is heavily loaded, that’ll slow things down even more. But, to a first approximation, we’ve got a stopwatch here. For more accuracy, you could write a script that relies on the %time% variable we discussed in Episode #49.

Time for Hal:

Emulating Ed’s loop is straightforward. I’ll even add a command at the end to pop up a window when time runs out:

$ for ((m=1; $m >= 0; m--)); do for ((s=59; $s >= 0; s--)); do \ echo $m minutes $s seconds to go; sleep 1; done; done; \ xterm -e 'echo TIME IS UP!; bash'

Notice that I’m executing bash in the xterm after the echo command. Spawning an interactive shell here keeps the xterm window from closing as soon as it echoes “TIME IS UP!”.

But Ed’s example got me thinking about ways to have a more accurate clock. One idea that occurred to me was to just use watch on the date command:

$ watch -dt -n 1 date

This will give you a clock that updates every second, with some extra highlighting to show the positions in the time string updating (that’s the “-d” option). But this isn’t a count down timer, it’s a “count up” timer.

Another idea I had was to use at or cron to pop up the warnings (see Episode #50 for more detail on at and cron). The only problem is that both at and cron are limited to 1 minute granularity, so they don’t work really well as a count down timer. But if you know your capture the flag session is supposed to end at noon, you could always do something like:

$ echo xterm -display $DISPLAY -e \'echo TIME IS UP\!\; bash\' | at noon

Notice I had to use a bunch of backwhacks so that the echo command passes a syntactically correct command into at. More interestingly, I have to explicitly declare my local $DISPLAY in the xterm command. While at is normally careful to preserve your environment variable settings, it apparently intentionally skips $DISPLAY– probably because there’s no guarantee you’ll actually be on that screen when you at job finally executes.

admin | Jul 28, 2009 | Comments 0

Prompts & Pushd

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Ed goes:

I was reading through our recent episodes the other day (a wonderful pastime activity that I encourage everyone to participate in). In Episode #49, Hal mentioned a nice option for incorporating the current time in the command prompt, a useful feature for forensics folks who want to record the time that they executed each command, as well as for general sysadmins and testers who want to see how long given activities take. In Episode #38, I mentioned how we could change our prompt by setting the environment variable called “prompt”, but I didn’t give a bunch of options there. Let’s explore these options in more detail, starting with including the time in our prompt a la Hal.

First off, instead of using the “set prompt=[whatever]” command to set our prompt to various values, we can alternatively use the prompt command to do so. The advantage of the latter approach is that it allows us to see all the glorious options we can use in setting our prompt:

C:\> prompt /?

By default, we’ve got $P for the path and $G for the greater than sign. To mimic Hal’s prompt from Episode #49, we could run:

C:\> prompt $C$T$F$G ( 8:02:19.57)> dir

That gives us a prompt of open paren ($C) followed by the time ($T) followed by close paren ($F) followed by a greater than sign.

For forensics guys, we may want to included the date as well, and remove the parens:

C:\> prompt $D$S$T$G Mon 07/20/2009 8:04:36.85>

Note that the $S is a space.

Now, these changes are just temporary, applying only to the current cmd.exe and any new cmd.exe processes it starts. To make the changes permanent, we need to alter the PROMPT environment variable in the registry

Mon 07/20/2009 8:14:05.60>reg add "hklm\system\currentcontrolset\control\session manager\environment" /v prompt /t reg_expand_sz /d $D$S$T$G The operation completed successfully.

After a reboot, your prompt will be changed for all users.

There is another interesting option we can add to our prompt via the $+ notation. This one will prepend our command prompt with a single plus sign for each directory we’ve pushed onto our directory stack via the pushd command. Wait a sec… before we get ahead of ourselves, let’s look at the pushd and popd commands a bit.

When working in a given directory, sometimes you need to temporarily change into another directory or two, do a little work there, and then pop back into the original directory you were working in. The pushd and popd commands were created to help make such transitions smoother. When changing directories, instead of using the familiar cd command, you could run “pushd [dir]” as in:

Mon 07/20/2009 8:18:02.21> prompt $P$G C:\Users\Ed> pushd c:\windows\system32 C:\Windows\System32>
This command will store your current directory in a stack in memory, and then change you to the other directory (in the case above, c:\Users\Ed will be stored on a stack, and you will change to c:\windows\system32). You can do stuff in that other directory, and even change to other directories beyond that. But, when your work is done there, you can go back to your original directory by simply running:

C:\Windows\System32> popd C:\Users\Ed>

While you can push a bunch of directories on this directory stack and then pop them off in a LIFO operation, I find the pushd/popd commands most useful for just storing a single directory I know I’ll have to pop back into in the near future, so I find myself often running:

C:\[wherever_I_am_working]> pushd .

Then, I do a little more work, and eventually change directories to where I need to work temporarily. When I’m ready to go back to where I was, I can simply popd. This technique is very helpful given that cmd.exe doesn’t have the “cd -” option found in bash so that I can simply change into a previous directory I was in earlier.

With that background of pushd and popd under our belts, we can now see what the $+ does in our prompt — It prepends one plus sign for each directory we’ve pushed.

C:\> prompt $+$P$G C:\> pushd c:\windows\system32 +C:\Windows\System32> pushd c:\Users\ ++C:\Users> pushd c:\temp +++C:\temp> popd ++C:\Users> popd +C:\Windows\System32> popd C:\>

The little pluses can help you remember how many things you have pushed on your directory stack.

And, if you wanted to get really elaborate and simulate the behavior of “cd -” by implementing some simple scripts to use in place of cd, you could do the following:

C:\> echo @pushd %1 > c:\windows\system32\cdd.bat C:\> echo @popd > c:\windows\system32\cd-.bat

Now, instead of using “cd” to change directories, you could always run “cdd” to do so (I use that last d to remind me that it’s pushing the directory onto the directory stack). Your new cdd command will work just like the old one, but it will remember the directories you’ve changed from, pushing them on the directory stack. Then, to go back to where you were before, you could run “cd-” (no space). It looks like this in action:

C:\> cdd c:\users\ed +C:\Users\Ed> cd- C:\>

Whither Ed goest, Hal will go!

Ed, you say you want the date in the prompt in addition to the time? So be it:

$ export PS1='\D{%D %T}> ' 07/20/09 10:02:50>
Unlike “\t” for the time code, there’s no built-in escape sequence for including the date in the prompt. But bash does recognize “\D{…}” to insert an arbitrary set of strftime(3) escape sequences. This is very flexible, though not quite as terse. I could even add the day name to fully emulate Ed’s Windows madness:

07/20/09 10:02:50> export PS1='\D{%a %D %T}> ' Mon 07/20/09 10:08:16>

You’ll also notice that Unix allows us to specify spaces as actual spaces rather than “$S”. What will those wacky Unix folks think of next?

Windows totally stole the pushd/popd idea from the Unix shell, and it works pretty much the same on both platforms:

$ export PS1='[\w]$ ' [~]$ pushd /tmp /tmp ~ [/tmp]$ pushd /usr/local/bin /usr/local/bin /tmp ~ [/usr/local/bin]$ popd /tmp ~ [/tmp]$ popd ~

You’ll notice that bash prints the directory stack after each pushd/popd operation, sort of as a reminder of where you are. You can also use the dirs command to dump the directory stack (in several different formats) or even clear the stack entirely:

[~]$ pushd /tmp /tmp ~ [/tmp]$ pushd /usr/local/bin /usr/local/bin /tmp ~ [/usr/local/bin]$ dirs # default /usr/local/bin /tmp ~ [/usr/local/bin]$ dirs -l # expand ~ to full pathname /usr/local/bin /tmp /home/hal [/usr/local/bin]$ dirs -p # one dir per line /usr/local/bin /tmp ~
[/usr/local/bin]$ dirs -c # clear the stack [/usr/local/bin]$ dirs /usr/local/bin

You can even use popd to selectively remove elements from the directory stack:

[/opt]$ dirs /opt /dev /usr/local/bin /tmp /usr /var /etc ~ [/opt]$ popd +2 /opt /dev /tmp /usr /var /etc ~

Notice that the elements of the directory list are numbered starting with zero (in C, arrays are numbered starting from zero and this convention was carried through to most Unix utilities), so “popd +2″ removes the third element counting from the left. “popd -2″ would remove the third element counting in from the right.

Unlike Windows, we don’t have to create a script file to make a cdd command on our Unix systems. We can just make an alias:

[~]$ alias cdd=pushd [~]$ cdd /tmp /tmp ~ [/tmp]$
What is interesting is that there’s no built-in Unix equivalent for the Windows “$+” prompting functionality. But since the bash shell does command substitution on $PS1 (if necessary) each time it emits a shell prompt, we can hack our own solution together:

[~]$ export PS1='`for ((i=1; $i < ${#DIRSTACK[*]}; i++)); do echo -n +; done`[\w]$ ' [~]$ pushd /etc /etc ~ +[/etc]$ pushd /tmp /tmp /etc ~ ++[/tmp]$ pushd /usr /usr /tmp /etc ~ +++[/tmp]$ popd /usr /etc ~ ++[/usr]$ popd /etc ~ +[/etc]$ popd ~ [~]$
As you can see, I’ve added an expression in backticks at the front of the declaration for $PS1, so whatever text this sequence of commands results in will be incorporated into the shell prompt. Inside the backticks, I’m using a for loop to produce the necessary plus signs. To terminate the loop, I’m comparing against “${#DIRSTACK[*]}” which translates to “the number of elements in the $DIRSTACK array variable”. $DIRSTACK is a highly magical environment variable that stores the elements of the directory stack that pushd and popd use.

You’ll also notice that I’m starting the loop at offset 1 rather than offset 0. As you can see in Ed’s Windows example, the “$+” element only counts the number of directories that have been explicitly pushed onto the stack with pushd. However, the bash $DIRSTACK variable always includes the current working directory. So when you pushd for the first time, the number of elements in $DIRSTACK is actually two: your original directory that you started in, plus the new directory you pushd-ed onto the stack. Anyway, I started my loop counter variable at 1 to more closely emulate the Windows behavior.

admin | Jul 21, 2009 | Comments 0

Leapin Lizards! ARP Stuff!

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

It’s Hal’s turn in the Bucket:

Mr. Bucket was pointing out the other day that we haven’t had much to say about viewing and manipulating the ARP cache from the command line, and that’s a darn shame. So let’s remedy that little oversight right now.

The ARP cache is the system’s local mapping of ethernet addresses to IP addresses of machines on the same LAN. On Unix systems, you can use “arp -a” to dump the cache or just dump information for a particular host:

$ arp -a gw.deer-run.com (192.168.1.1) at 00:04:23:5f:20:98 [ether] on eth0 deer.deer-run.com (192.168.1.2) at 00:30:48:7b:22:2e [ether] on eth0

$ arp -a gw gw.deer-run.com (192.168.1.1) at 00:04:23:5f:20:98 [ether] on eth0

As with most network administration commands on Unix, the “-n” flag suppresses the hostname information, in case you’re having name resolution problems that are causing the command to hang:

$ arp -an ? (192.168.1.1) at 00:04:23:5f:20:98 [ether] on eth0 ? (192.168.1.2) at 00:30:48:7b:22:2e [ether] on eth0

Occasionally monitoring your ARP cache can be interesting, because it can alert you when system hardware is changing on your network and also to ARP spoofing attacks. Here’s a little bit of shell fu to watch for changes to a particular ARP entry:

$ while :; do c=`arp -a 192.168.1.1 | awk '{print $4}'`; \ [ "X$c" == "X$p" ] && echo "$c (OK)" || echo "$p to $c (CHANGED)"; \ p=$c; sleep 5; done to 00:04:23:5f:20:98 (CHANGED) 00:04:23:5f:20:98 (OK) 00:04:23:5f:20:98 (OK) ^C

Notice the equality test where I’m putting an “X” in front of the value of each variable (“X$c” == “X$p”)? That’s a common trick for doing comparisons where one of the values might be null– as in our case when we first enter the loop and $p is not set yet. If we just wrote, “[ $c == $p ]“, the shell would generate a syntax error message during the first pass through the loop because it would interpret our fu as “[ 00:04:23:5f:20:98 == ]” which is clearly bogus. The extra “X” ensures that neither side of the comparison is ever empty.

Actually, the above example is a little bogus (though it includes plenty of tasty shell fu), because there’s already a tool called Arpwatch that will monitor your ARP cache for changes– expected or otherwise. I’ll often run this tool on a couple of machines on critical networks, just to look for signs of hanky panky.

If you’re running with root privileges, you can also use the arp command to manually manipulate your ARP cache. For example, “arp -d” deletes ARP entries. However, since ARP dynamically relearns the deleted entries, they don’t tend to stay deleted for long:

# arp -d deer # arp -a gw.deer-run.com (192.168.1.1) at 00:04:23:5f:20:98 [ether] on eth0 deer.deer-run.com (192.168.1.2) at on eth0 # ping -c 1 deer >/dev/null # arp -a gw.deer-run.com (192.168.1.1) at 00:04:23:5f:20:98 [ether] on eth0 deer.deer-run.com (192.168.1.2) at 00:30:48:7b:22:2e [ether] on eth0

The root account can also use “arp -s” to add static ARP assignments to the cache. For example, I was recently debugging a VPN problem for one of my customers and we suspected the Open Source VPN software we were running on our Unix system wasn’t properly doing Proxy ARP for the remote clients. So we manually added an ARP entry for the client we were testing with that pointed to the MAC address of our gateway (note that you must do this on the gateway machine that is the owner of the specified MAC address):

# arp -s 192.168.1.202 00:04:23:5f:20:98 pub # arp -an ? (192.168.1.2) at 00:30:48:7b:22:2e [ether] on eth0 ? (192.168.1.202) at PERM PUB on eth0
The extra “pub” argument means that the host should “publish” this static ARP entry– that is, respond with our local MAC address when other hosts on the LAN make ARP requests for 192.168.1.202. In other words, our gateway should perform Proxy ARP for this address.

Since we did not specify the “temp” option in the above command, this static ARP entry will persist until we reboot the machine. If you specify “temp”, then the entry will time out as normal when your system flushes its ARP cache. Generally, though, if you’re specifying static ARP entries, you’re doing it for a reason and you want them to stick around. For example, some highly secure sites will populate static ARP entries for all authorized hosts on their local LANs in order to thwart ARP spoofing attacks (static ARP entries will never be overwritten by dynamic ARP information learned from the network). Of course, the static ARP entries will need to be reloaded every time the system reboots, so you’ll need to create a boot script to automatically reload the entries from a configuration file. And you’d need to update the configuration file every time a host was added or a NIC was changed on the LAN. It’s a huge hassle and not normally done in practice.

That’s about it for ARP on Unix. If memory serves, things aren’t too different on Windows, but I bet Ed has some tasty fu for us anyway…

Ed responds to Hal’s Bucket Screed:

As Hal anticipated, the arp command on Windows is very similar to the Linux version, but there are a few annoying deltas. To dump the ARP cache, we can run:

C:\> arp -a

Interface: 10.10.10.15 --- 0x10003 Internet Address Physical Address Type 10.10.10.45 00-0c-29-c2-9f-1b dynamic 10.10.11.11 de-ad-be-ef-00-00 static 10.10.75.1 00-0c-29-a3-2c-8b dynamic
There are a couple of noteworthy things here: First off, we don’t see host names. The Windows arp command works exclusively with IP addresses and not names. Thus, we don’t have a -n option, and cannot enter domain names of the machines at command line. If we want to zoom in on individual machines, we need to enter an IP address, not a name:

C:> arp -a 10.10.11.11

Interface: 10.10.10.15 --- 0x10003 Internet Address Physical Address Type 10.10.11.11 de-ad-be-ef-00-00 static

Next, note that we have that “Type” column, which tells us whether an entry is dynamic or static. This is handy, because we can list all static ARP entries using:

C:\> arp -a | find "static"

Next, note that the Windows arp command uses dashes between octets in the MAC address, and not colons. This is annoying, and I often type it wrong before I type it right.

Hal’s ARP-monitoring command is fun and useful, so I tried to mimic its functionality almost identically on Windows, coming up with the following command (brace yourselves):

C:\> cmd.exe /v:on /c "for /l %i in (1,0,2) do @for /f "skip=3 tokens=2" %i in ('arp -a 10.10.10.45') do @set current=%i & (if X!current!==X!previous! (echo %i ^(OK^)) else echo !previous! to %i ^(CHANGED^)) & set previous=%i & ping -n 6 127.0.0.1 > nul)"

!previous! to 00-0c-29-c2-9f-1b (CHANGED) 00-0c-29-c2-9f-1b (OK) 00-0c-29-c2-9f-1b (OK) 00-0c-29-c2-9f-1b (OK) 00-0c-29-c2-9f-1b (OK) 00-0c-29-c2-9f-1b to 01-02-03-04-05-06 (CHANGED) 01-02-03-04-05-06 (OK) 01-02-03-04-05-06 (OK) 01-02-03-04-05-06 (OK)

To analyze this command, let’s work our way from the outside in. I’m invoking a cmd.exe with delayed environment variable expansion so that I can store and compare some variables inside my command. I then have a FOR /L loop that counts from 1 to 2 in steps of zero to keep this puppy running forever. I add a 5-second delay at the end by pinging 127.0.0.1 six times. Now, we get to the core of the command. I run a FOR /F loop to parse the output of my arp command, skipping 3 lines of output to zoom in on what I want, grabbing the second item (the MAC address), which I place into iterator variable %i. At each step through the loop, I set the variable “current” to my iterator variable so I can compare it against my previous MAC address for that ARP table entry with an IF statement. IF statements don’t like to deal with FOR loop iterator variables, so I have to move my %i value into “current” for comparison. I then compare it with the previous value, using Hal’s little X prepend trick so they are never empty strings. If they match, I print out the MAC address (%i) and the “(OK)” just like Hal does. I have to put ^’s in front of my parens so that echo knows to display them. If the current and previous MAC address from that arp entry do not match, I print out that the previous changed to %i followed by “(CHANGED)”. Finally, I store the current value of %i as my “previous” variable and then wrap around. It’s not elegant, but it gets the job done, and looks very similar to the output and functionality of Hal’s command.

Lessee… what other stuff does Hal have for us?

Yup… we have “arp -d” to delete an ARP entry:

C:\> arp -d 10.10.10.45
And, we also have arp -s to plant a static ARP entry:

C:\> arp -s 10.10.10.45 01-02-03-04-05-06

Unfortunately, the Windows arp command does not have a “pub” option like the Linux command to implement proxy ARP functionality.

By the way, this whole discussion of the arp command was triggered by Byte Bucket’s comment to Hal and me in our discussion of scheduling tasks. Byte (if I can be so informal) mentioned that on Windows, to define a static ARP entry, Microsoft recommends that you schedule a startup task, by saying, “To create permanent static ARP cache entries, place the appropriate arp commands in a batch file and use Scheduled Tasks to run the batch file at startup.” This is a good idea if you ever want to hard code your arp tables to avoid ARP cache-poisoning attacks. Of course, in their recommendation, Microsoft never tells you how to actually do this. Well, never fear… Command Line Kung Fu is here!

We start by creating our batch file for the static ARP entry:

C:\> echo arp -s 10.10.10.45 aa-bb-cc-dd-ee-ff > C:\static_arp.bat

You can add more static arp entries into this file if you’d like, echoing in one arp command per line, appending to the file with >>. I’m storing it right in c:\ itself for easy spotting, although you may find that tucking it away somewhere you keep your admin scripts is a less clutterful approach. Then, we make that a startup task using the schtasks command, which we described in our last episode (#50):

C:\> schtasks /create /sc onstart /tr c:\static_arp.bat /tn static_arp

And there you have it! We’ve seen how to make a little arp-cache watcher in a single command, as well as how to implement static arp entries at system startup. Good topic, Hal!

admin | Jul 14, 2009 | Comments 0

Environment-A-List

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Ed starts out:

At a cmd.exe prompt, you can list all environment variables using the “set” command by itself:

C:\> set

Pretty simple. By default, there are a typically a lot of variables set, upwards of two or three dozen.

Another property of set involves the listing of groups of environment variables that start with a substring. For example, to see all of your environment variables that start with the letters pr, you could type:

C:\> set pr

It’s kinda weird, but environment variables in Windows cmd.exe are case insensitive.

We can refer to any one of these variables in a command by using the %[var]%. Thus, we can see the value of the username variable by running:

C:\> echo %username%

Or, we can see the value using the set command with:

C:\> set username

Prior to Vista, Windows didn’t include a built-in whoami command. The closest thing we have is this “set username”. However, be aware that, depending on how your shell is instantiated, you may or may not have this variable set. For example, this variable is typically not set when Metasploit launches a shell on a compromised box. Windows Vista does include a whoami command, but that’s a story for another article. We must keep focus when performing command line kung fu.

Other environment variables besides username that are worthwhile include our path variable:

C:\> set path Path=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH

The output here will show you both your path, plus the pathext variable (remember, set [string] shows all variables that start with string). The pathext variable is particularly interesting, because it tells us the priority of execution on the machine for various file name suffixes. By default, if I type in “command”, but don’t specify a suffix, command.com will be run first. If that doesn’t exist, the system tries command.exe. Then, it moves onto command.bat and so forth. That behavior can be used for some evil stuff by creating a backdoor called cmd.com or ipconfig.com and inserting it somewhere in the user’s path.

Windows, by the way, includes an implicit “.” at the very beginning of every user’s path. That’s a huge bummer from a security perspective, because users can be tricked into running a backdoor out of their current directory when they try to invoke some other command. Also, note that Windows doesn’t seem to be too proud that “.” is in the path, because it doesn’t show it… but, it behaves as though it’s there allright.

Other useful environment variables include %systemroot%, which tells you which partition and folder your primary Windows software is located in:

C:\> echo %systemroot%
Often when I’m handling an incident, I see attackers who assume the system root is on C:\. But, sometimes, it’s not, and attackers get wrapped around the axel trying to figure out why their tweaks to the OS aren’t having an impact. Whenever I’m doing a penetration test and I compromise a Windows box, I quickly display my systermroot and username variables, because that helps to orient me to the target machine.

Also, the prompt environment variable is called, simply enough, prompt. It’s default value is $P$G, which indicates your current working directory ($P) followed by a greater than sign ($G). You could really freak yourself out by running:

C:\> set prompt=$$ $ ls

You now have a Linux wanna-be for your shell. I tell ya, when I see that $ prompt, and I just want to type “ls”. Alas, no dice. :)

Hal responds:

It’s interesting to me how even something as simple as environmental variables points out the differences between the Windows and Unix command-line philosophies. For example, instead of packing all kinds of functionality into a single command like Windows does with “set”, Unix handles this by providing a toolkit of different programs that all interact with each other. For example, you use the “env” command to dump all of your current environment variable settings. If you want a specific subset, you pipe the output of “env” into “grep”:

$ env | grep PATH MANPATH=/usr/man:/usr/share/man:/usr/openwin/man:/usr/dt/man:/usr/local/man CDPATH=.:/home/hal LD_LIBRARY_PATH=/usr/lib:/usr/openwin/lib:/usr/dt/lib:/usr/local/lib PATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:...

This seems cumbersome– you have to use two commands and a pipe instead of using just one “set” command with different parameters. But the power of this idiom is that you use the same knowledge of “grep” to search any kind of textual data in Unix, whereas under Windows you have to learn the particular idiosyncracies of a whole host of separate command line interfaces that each only deal with very specific types of data.

Anyway, returning to Ed’s examples, you can display the value of a particular variable using “echo”, and we typically set new values using “export” so that the variable setting will be passed along to any subshells we spawn:

$ echo $PATH PATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:... $ export PATH=.:$PATH $ echo $PATH PATH=.:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:... $ export PATH=${PATH/\.:/} $ echo $PATH PATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:...

Here I’m using the current value of $PATH in the “export” expression so that I can prepend “.” (the current working directory) at the front of my search path. Obviously, as Ed points out above, this is a terrible idea from a security perspective, so I then use the same variable substitution operator we used back in Episode 26 to remove the leading dot. Again, I’m leveraging prior knowledge of basic shell building blocks to solve a new problem.

As Ed points out, Unix does have the “whoami” command, plus a number of other mechanisms for figuring out what user you are that provide different types of information:

$ whoami hal $ who am i hal pts/2 2009-04-26 14:44 (:0.0) $ id uid=1000(hal) gid=1000(hal) groups=4(adm),20(dialout),24(cdrom),...

Also, the your login program will generally set the $LOGNAME and $HOME environment variables, and your shell may also set the $USER and/or $USERNAME variables as well.

You can change your prompt by setting the $PS1 environment variable. For example, the default on my Ubunty machine is:

$ export PS1='\u@\h:\w\$ ' hal@elk:~$
There are a whole host of different escape sequences you can use here: “\u” is your user ID, “\h” is the unqualified host name, and “\w” is the current working directory. So if we wanted to emulate the standard Windows command prompt in our Unix shell we could use:

hal@elk:~$ export PS1='\w> ' ~> cd /tmp /tmp>
Of course, it doesn’t look exactly like the Windows prompt, because when you’re in your home directory, the directory is listed as “~”, not $HOME. However, you can use the magic $PWD variable in your prompt instead of “\w” to get a more Windows-like experience:

/tmp> export PS1='$PWD> ' /tmp> cd /home/hal>

The emulation isn’t perfect because Unix uses forward slashes in directory paths, not backslashes. I present the following for the truly psychotic:

/home/hal> export PS1='C:${PWD//\//\\\\}> ' C:\home\hal> cd /tmp C:\tmp>

You think having a “$” prompt will confuse the Windows folks? Try putting the above recipe into your /etc/profile file next April Fool’s Day and watch your Unix users freak out.

By the way, you’ll notice that the variable is $PS1. There’s also $PS2:

C:\home\hal> export PS1='$ ' $ export PS2='more> ' $ for i in * more> do more> ...

Believe it or not, there are even $PS3 (the prompt used by the “select” command) and $PS4 (usually set to “+”, this is the value printed before each command in an execution trace as with “set -x”).

Paul Throws His Hat In:

One of the environment variables that I’ve encountered in the past that has really helped me is “LD_LIBRARY_PATH”. The variable stores the path that will be used by programs to find software libraries. For example, if you’ve got a bunch of custom libraries in /opt/local/lib you could run the following command:

$ export LD_LIBRARY_PATH="/opt/local/lib:/usr/local/lib:/usr/lib:/lib"

This comes in handy when you are using OS X and something such as MacPorts is installed in /opt and has the libraries you are looking for. Yes, these are in fact the libraries you are looking for…

admin | Apr 27, 2009 | Comments 0