All Entries Tagged With: "file"

Searching Text Strings

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal checks out the mail

We love getting email from readers of the blog. And we love getting cool shell hacks from readers even more. Recently, loyal reader Rahul Sen sent along this tasty little bit of shell fu:

How to search for certain text string in a directory and all its subdirectories, but only in files of type text, ascii, script etc:

$ grep 9898 `find /usr/local/tripwire -type f -print | xargs file | egrep -i 'script|ascii|text' | awk -F":" '{print $1}'` /usr/local/tripwire/te/agent/data/config/agent.properties:tw.local.port=9898 /usr/local/tripwire/te/agent/data/config/agent.properties:tw.server.port=9898

That’s totally cool, Rahul!

Honestly, when I first looked at this I thought, “There’s got to be a shorter way to do this.” But the tricky part is the “only in files of type text, ascii, script, etc” requirement. This basically forces you to do a pass through the entire directory first in order to locate the relevant file types. Thus the complicated pipeline to pass everything through the file command and egrep.

A few minor improvements I might suggest:

1. I’m worried that for a large directory you’ll end up returning enough file names that you exceed the built-in argument list limits in the shell. So it might be better to use xargs again rather than backticks.

2. I probably would have chosen to use sed at the end of the pipeline rather than awk, just to be more terse.

3. You don’t actually need “-print” on modern versions of find– it’s now the default. Only old-timers like me and Rahul end up doing “-print” all the time because we were trained to do so by old versions of find.

So my revised version would look like:

Mmmm-hmmm! That’s some tasty fu! Let’s see what my Windows brethren have cooking…

Ed Unfurls:

This is a helpful little technique. Now, unfortunately at the Windows command line (man, if I only had a dime ever time I said that phrase), we do not have the “file” command to discern the type of file. But, fear not! We do have a couple of alternative methods.

For a first option, we could use a nifty feature of the findstr command to ignore files that have non-printable characters. When run with the /p option, findstr will ignore any files that contain high-end ASCII sequences, letting us skip over EXEs, DLLs, and other stuff. It’s not as fine a grained scalpel as scraping the output of the the Linux file command for script, ascii, and text, but it’ll serve us well as follows:

C:\> findstr /s /p 9898 *

Here, I’m using findstr to recurse the file system (/s) from wherever my current working directory is, skipping files with non-printable characters (/p), looking for the string 9898 in any file (*). If you want to get even closer to the original, we can specify a directory where we want to start the search using the /d: option as follows:

C:\> findstr /d:C:\windows /s /p 9898 *

Now, for our second option, there is another way to refine our search besides the /p option of findstr, getting us a little closer to the file types Rahul specified in Linux using the find command. It turns out that Microsoft actually put an indication of each file’s type in the name of the file itself. You see, by convention, Windows file names have a dot followed by three letters that indicate the file type. Who knew?!?! :)

To map the desired functionality to Windows, we’ll rely on file name suffixes to look inside of .bat, .cmd, .vbs, and .ps1 files (various scripts), .ini files (which often contain config info), and .txt files (which, uh… you know). What’s more, many commands associated with searching files allow us to specify multiple file names, with wild cards, such as the dir command in this example:

C:\> dir *.bat *.cmd *.vbs *.ps1 *.ini *.txt

And, happy to say, dir isn’t the only one that lets us look for multiple file names with wildcards. For my second solution to this challenge, I’m going to use a FOR /R loop. These loops recurse through a directory structure (/R, doncha know) setting an iterator variable to the name of each file that is encountered. Thus, we can use the following command as a rough equivalent to Rahul’s Linux fu:

C:\> FOR /R C:\ %i in (*.bat *.cmd *.vbs *.ps1 *.ini *.txt) do @findstr 9898 "%i" && echo %i

Here, I’m running through all files found under C:\ and its subdirectories, looking inside of any file that has a suffix of .bat, .cmd, etc, running findstr on each file (whose name is stored in %i, which has to be surrounded in quotes for those cases when the value has one or more spaces in the file name) looking for 9898. And, if I successfully find a match, I echo out the file’s name. Now, this output looks a little weird, because the file’s name comes after each line that contains the string. But, that is a more efficient way to do the search. Otherwise, I’d have to introduce unnecessary complexity by using a variable and parsing to store the line of the file and print its name first, then print the contents. I’d certainly do that for prettiness in a script. But, at the command line by itself, I’d eschew the complexity and just go with what I’ve shown above to get the job done.

Now, there’s a gazillion other ways to do this as well. For a third possibility, we could take the first option above (findstr) and use the multiple file suffix specification of option 2 (*.bat *.cmd *.vbs *.ps1 *.ini *.txt) to come up with:

C:\> findstr /d:C:\windows /s 9898 *.bat *.cmd *.vbs *.ps1 *.ini *.txt

I actually like this third approach best, because it’s relatively easy to type, makes a bunch of sense, has nicer-looking output than the FOR /R option, and has better performance.

Fun, fun, fun! Thanks for the great suggestion, Rahul.

Whatcha got for us, Tim?

Tim delivers:

Sadly, PowerShell is missing the same “file” command as the standard Windows command line. Also, there isn’t a PowerShell cmdlet similar to “findstr /p” either, but of course we could use FindStr since all the Windows commands are available in PowerShell. Ed already covered FindStr so we will just use just PowerShell cmdlets.

If we know that the files in question are in a specific directory, not subdirectories, there is a pretty simple command to find the files using the Select-String cmdlet.

PS C:\> Select-String 9898 -Path *.bat,*.cmd,*.vbs,*.ps1,*.ini,*.txt -List | Select Path

Path ---- C:\temp\a.txt

According to the documentation “the Select-String cmdlet searches for text and text patterns in input strings and files. You can use it like Grep in UNIX and Findstr in Windows.” Whoah, big difference there! Grep has much more robust regular expression support when compared to FindStr, and yes, PowerShell does give us rich regular expressions.

Back to the task at hand. We only care if the file contains the text in question, not how many times the text is found in the file. The List parameter is used as a time saver, since it will stop searching after it finds the first match in a file.

For each match, the default console output displays the file name, line number, and all text in the line containing the match. Of course the output is an object and we just want the file’s path, so the results are piped into Select-Object (alias select) in order to get the full file path.

But we want to search though subdirectories too. To do that we have to use Get-ChildItem (alias dir, gci, ls).

PS C:\> Get-ChildItem -Include *.bat,*.cmd,*.vbs,*.ps1,*.ini,*.txt -Recurse | Select-String 9898 -List | Select-Object path

Path ---- C:\temp\subfolder\b.txt C:\temp\a.txt

The Recurse parameter specifies that the search should recursively search through subdirectories. The Include parameter retrieves only the files that match our filter. We could use the Exclude parameter if we wanted to search all files that aren’t exe’s or dll’s.

PS C:\> Get-ChildItem -Exclude *.exe,*.dll -Recurse | Select-String 9898 -List | Select-Object path

The command can be shortened even further since the full parameter name doesn’t have to be used. As long the shortened parameter name isn’t ambiguous a short version of the parameter name can be used. Since there is no other parameters that start with R we or I this command will work as well:

PS C:\> ls -i *.bat,*.cmd,*.vbs,*.ps1,*.ini,*.txt -r | select 9898 -L | select path

There is a catch if you are using version 1 of PowerShell, the Select-String cmdlet natively doesn’t take the input from Get-ChildItem and use it to specify the path. We have to use a For-EachObject loop (alias %) in order to accomplish the same task.

PS C:\> Get-ChildItem -Include *.bat,*.cmd,*.vbs,*.ps1,*.ini,*.txt -Recurse | % { Select-String 9898 -List -Path $_.FullName } | Select-Object path

A side note:
As you probably already know, Windows XP, Vista, and 2003 don’t come with Powershell and require a separate install, but for the love of Pete, install it. Version 2 has been available for quite a while and there are many enhancements over v1 (and even more when compared to cmd). Windows 2008 R2 and Windows 7 come with v2 by default.

PowerShell v1 in Windows 2008 (R1) is an optional feature that needs to be enabled. It can be installed using the Windows command line by running this command:

C:\> ServerManagerCmd.exe -install PowerShell

This is probably the most useful Windows command available (sorry Ed).

admin | Feb 09, 2010 | Comments 0

A Sort of List

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal starts off:

Way back in Episode #11 I showed you a little trick for sorting directory listings by inode number. But it struck me recently that we hadn’t talked about all of the other interesting ways you can sort directory listings.

For example, you can use “ls -S” to sort by file size:

$ ls -lS total 6752 -rw-r----- 1 syslog adm 1271672 2010-01-18 05:36 kern.log.1 -rw-r----- 1 syslog adm 1016716 2010-01-18 05:39 messages.1 -rw-r----- 1 syslog adm 499580 2010-01-18 05:38 daemon.log.1 [...]

Add “-h” if you prefer to see those file sizes with human-readable units:

$ ls -lSh total 6.6M -rw-r----- 1 syslog adm 1.3M 2010-01-18 05:36 kern.log.1 -rw-r----- 1 syslog adm 993K 2010-01-18 05:39 messages.1 -rw-r----- 1 syslog adm 488K 2010-01-18 05:38 daemon.log.1 [...]

Also, adding “-r” (reverse sort) can be useful so that the largest files end up at the bottom of the directory listing, closer to your next command prompt:

$ ls -lShr total 6.6M [...] -rw-r----- 1 syslog adm 488K 2010-01-18 05:38 daemon.log.1 -rw-r----- 1 syslog adm 993K 2010-01-18 05:39 messages.1 -rw-r----- 1 syslog adm 1.3M 2010-01-18 05:36 kern.log.1 $

You have to do much less scrolling around this way.

In addition to sorting by size, you can also sort by the so-called “MAC time” values: last modified (mtime), last access (atime), and last inode or meta-data update (ctime). By default, “ls -t” will sort by last modified time. This is another good one to use “-r” on so you can quickly find the most recently modified files in a directory:

$ ls -lrt total 6752 [...] -rw-r----- 1 syslog adm 86080 2010-01-18 08:10 kern.log -rw-r----- 1 syslog adm 120492 2010-01-18 08:17 syslog -rw-r----- 1 syslog adm 3310 2010-01-18 08:17 auth.log $

If you want to sort by ctime you use “-c” in addition to “-t”. However, to sort by atime you need to use “-u” (“-a” was reserved for something else, obviously):

$ ls -lrtu total 6752 [...] -rw-r--r-- 1 root root 219990 2010-01-18 08:00 udev -rw-r--r-- 1 root root 120910 2010-01-18 08:00 Xorg.0.log -rw-r----- 1 root adm 56275 2010-01-18 08:00 dmesg $

Now let’s see what my Windows brethren have up their sleeves, shall we?

Ed Responds:

Although not as full featured as the Linux ls command, the humble dir command offers us a bunch of options, allowing us to mimic pretty much everything Hal has done above. The main options we’ll use here are:

* /o followed by a one-character option that lets us specify a sort order (we’ll use /os to sort by size and /od by date… with a – sign in front of the one character to reverse order)
* /t, also followed by one character which lets us specify a time field we’re interested in (the field options we have and their definitions, according to the dir command’s help, are /tc for Creation time, /ta for Last Access time, and /tw for Last Written time).

So, to get a directory listing sorted by size (smallest to largest), we’d run:

C:\> dir /os

Want them reversed? We would use:

C:\> dir /o-s

Want those sizes in human readable form? Install Cygwin and use the ls command, for goodness sakes. This is the dir command we’re talking about here. We don’t need no stinkin’ human readable format. Actually, the default output for dir does show commas in its size numbers, making things a little more readable than the stock Linux output.

To see directory contents listed by Last Written (which is what dir calls them… roughly the same as last modified times in Linux parlance), in reverse order (with the most recently modified near the top), you could execute:

C:\> dir /o-d /tw

But, like we see with the ls command, Last Written is the default, so you can leave off the /tw to get the same results.

Wanna sort by creation time, again in reverse? Use:

C:\> dir /o-d /tc

And, how about last access? You could go with:

C:\> dir /o-d /ta

It’s a good thing that the /od and /o-d sort options pick up the proper timestamp specified by the /t option, or else we’d be forced to do some waaaaay ugly sort command nonsense. Whew!

Tim responds too:

To get a directory listing we use Get-ChildItem. The name is a bit odd, but it is a generic command and can be used to get the child items from any container such as the registry, file system, or the certificate store. Today we are just looking at the file system.

First, let’s take a look at the aliases for this useful cmdlet.

PS C:\> Get-Alias -Definition Get-ChildItem

CommandType Name Definition ----------- ---- ---------- Alias dir Get-ChildItem Alias gci Get-ChildItem Alias ls Get-ChildItem

I typically use ls since it is 33% more efficient to type than dir. But I digress…

Let’s sort by file size:

PS C:\> gci | sort length

Directory: C:\

Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- 6/10/2009 4:42 PM 10 file1.txt -a--- 6/10/2009 4:42 PM 24 file2.txt -a--- 11/24/2009 3:56 PM 1442522 file3.zip

The Get-ChildItem cmdlet does not have sorting capability built in, none of the cmdlets do. But that is what the pipeline and the Sort-Object cmdlet are for.

Want to sort by file size in reverse order? Use the Descending parameter.

PS C:\> gci | sort length -descending

Directory: C:\

Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- 11/24/2009 3:56 PM 1442522 file3.zip -a--- 6/10/2009 4:42 PM 24 file2.txt -a--- 6/10/2009 4:42 PM 10 file1.txt

We can sort by any property, including LastAccessTime, LastWriteTime, or CreationTime.

PS C:\> gci | sort LastWriteTime

We can even sort on two properties.

PS C:\> gci | sort LastWriteTime, Length

Directory: C:\

Mode LastWriteTime Length Name ---- ------------- ------ ---- -a--- 6/10/2009 4:42 PM 24 file2.txt -a--- 6/10/2009 4:42 PM 10 file1.txt -a--- 11/24/2009 3:56 PM 1442522 file3.zip

The files will first be sorted by write time. If two files have the same write time, they will then be sorted by length.

Finally, we come to displaying the size in a human readable format, and it isn’t pretty. We have to write a custom expression to display the size in KB or MB.

PS C:\> gci | format-table -auto Mode, LastWriteTime, Length, @{Name="KB"; Expression={"{0:N2}" -f ($_.Length/1KB) + "KB" }}, @{Name="MB"; Expression={"{0:N2}" -f ($_.Length/1MB) + "MB" }}, Name

Mode LastWriteTime Length KB MB Name ---- ------------- ------ -- -- ---- -a--- 6/10/2009 4:42 PM 10 0.01KB 0.00MB file1.txt -a--- 6/10/2009 4:42 PM 24 0.02KB 0.00MB file2.txt -a--- 11/24/2009 3:56 PM 1442522 1,408.71KB 1.38MB file3.zip

We can specify custom properties to display. This format works with any of the format cmdlets (Get-Command -Verb Format) or select-object. The custom columns are created by using a hashtable. A hashtable is specified by using @{ key1=value1, key2=value2 }. In our case we specify a name and an expression. Here is a simple example.

..., @{Name="Foo"; Expression={ $_.Length + 1 }}, ...

In this case we would add a column with the heading Foo and with a value of the Length plus 1. The expression can include all sorts of math or other crazy PowerShell fu.

Ironically, getting a human readable output comes from a non-human readable command.

admin | Jan 26, 2010 | Comments 0

Joining Up

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal fields a question from IRC

Mr. Bucket passed along the following query from the PaulDotCom IRC channel:

What functionality is available to loop through multiple files, and write the output to a single file with some values on the same line? Ex: If one program gives me the hash of a file, and the other program outputs the name/size/etc of a file, can I output to the same file HASH-FileName-Size

I couldn’t resist chortling with glee when this question came up, because it’s another one of those “easy for Unix, hard for Windows” kinds of tasks. I never can resist sharing these “learning experiences” with my fellow co-authors.

First let’s review our inputs. I’m going to use the openssl utility for generating checksums, since it’s fairly generic to lots of different flavors of Unix at this point:

$ openssl sha1 * SHA1(001.jpg)= a088531884ee5eb520e98b3e9e18283f29e13d25 SHA1(002.jpg)= 77febb1498b2926ee6a988c97f3457e38736456d SHA1(003.jpg)= 922bcb001d025d747c2ee56328811a4270b62079 ...

As you can see, it’s pretty easy to generate a set of checksums over my directory of image files, but there’s a bunch of cruft around the filename that’s not really helpful. So let me get rid of that with some quick sed action:

$ openssl sha1 * | sed -r 's/SHA1$(.*)$= (.*)/\1 \2/' 001.jpg a088531884ee5eb520e98b3e9e18283f29e13d25 002.jpg 77febb1498b2926ee6a988c97f3457e38736456d 003.jpg 922bcb001d025d747c2ee56328811a4270b62079 ...

That’s better! In the sed expression I’m using the “(.*)” sub-expressions to match the file name and the checksum in each line, and the substitution operator is replacing the original line with just the values of the sub-expressions. Slick.

Now that we’ve got the checksums, how do we produce the file sizes? I could just use “ls -l” of course. But since the questioner seems to only want “HASH-FileName-Size”, I may as well just use “wc -c” to produce simpler output:

$ wc -c * 4227504 001.jpg 4600982 002.jpg 4271719 003.jpg ...
Now that I know what my inputs are going to be, the question is how to stitch them together? Luckily, Unix includes the join command for putting files together on arbitrary fields (we last saw the join command back in Episode #43). Now I could save the checksum output and the file sizes to separate files and then join the contents of the two files, but bash actually gives us a cooler way to handle this:

$ join -1 1 -2 2 <(openssl sha1 * | sed -r 's/SHA1$(.*)$= (.*)/\1 \2/') <(wc -c *) 001.jpg a088531884ee5eb520e98b3e9e18283f29e13d25 4227504 002.jpg 77febb1498b2926ee6a988c97f3457e38736456d 4600982 003.jpg 922bcb001d025d747c2ee56328811a4270b62079 4271719 ...

See the “<(…)” syntax? That’s a little bit of bash file descriptor magic that allows us to substitute the output of a command in a place where a program would normally be looking for a file name. In this case it saves us the hassle of having to create intermediate output files to join together. The join command itself is pretty simple. We’re telling the program to join the output of the two commands using the file names in the first field of input #1 and the second field of input #2. The only problem is that the join command isn’t producing the “HASH-FileName-Size” output that the original questioner wanted. That’s because join always outputs the joined field first, followed by the remaining fields from the first input (the checksum in this case), followed by the remaining fields from the second input (the file size). We’ll have to use a little awk fu to re-order the fields:

$ join -1 1 -2 2 <(openssl sha1 * | sed -r 's/SHA1$(.*)$= (.*)/\1 \2/') <(wc -c *) \ | awk '{print $2 " " $1 " " $3}' a088531884ee5eb520e98b3e9e18283f29e13d25 001.jpg 4227504 77febb1498b2926ee6a988c97f3457e38736456d 002.jpg 4600982 922bcb001d025d747c2ee56328811a4270b62079 003.jpg 4271719

… Mmmm, that’s a tasty little bit of shell magic, isn’t it? Let’s see what Ed and Tim are cooking up.

Ed retorts snidely:

Choosing a topic just because you think it’s hard for us Windows guys, huh, Hal? Well, aren’t you just a big ball of sunshine, a command-line Scrooge this holiday season? When I first read this one, I though… “Ugh… this is gonna be hard.” Perhaps I was psyched out by your juvenile trash talk. Or, maybe I’ve just been hanging around in cmd.exe too long, and have gotten used to hard problems.

But, this one turned out to be surprisingly straight-forward and even non-ugly (well, beauty is in the eye of the beholder, I suppose). Here’s the skinny:

C:\> FOR /f "tokens=1-2" %a in (name-hash.txt) do @for /f "tokens=1,2" %m in (length-name.txt) do @if %a==%n echo %b %a %m a088531884ee5eb520e98b3e9e18283f29e13d25 001.jpg 4227504 77febb1498b2926ee6a988c97f3457e38736456d 002.jpg 4600982 922bcb001d025d747c2ee56328811a4270b62079 003.jpg 4271719

I’m assuming that name-hash.txt contains, well, names and hashes, one pair per line. Likewise, length-name.txt contains lengths and names, again one pair per line.

As we know, FOR /F loops can parse through all kinds of crap, including the contents of files. I use a FOR /F loop with two tokens (giving me two variables) of %a (for the file name) and %b (allocated automagically, holding the hash). For each of the files described in name-hash.txt, I then construct the body of my FOR loop. It contains another FOR /F loop, again with two variables (the original question mentioned “etc” for extra stuff there… if you have more stuff, just up the number of tokens and echo the proper variables at the end). My inner FOR /F loop iterates through the length-name.txt file, placing its values in the variables %m (length) and %n (name).

Now, if I just echoed out %a %b %m %n, I’d be making all of the possible combinations of every pair of two lines in the original files. But, we want to pare that down. We only want to generate some output if the name from name-hash.txt (%a) matches the name from length-name.txt (%n). We do this with a little IF operation comparing the two variables. If they match, we then echo out hash (%b), name (%n), and size (%m).

Admittedly, the performance of this little command isn’t great, as I have to run through every line of name-hash.txt, comparing the name by running through the entirety of length-name.txt. I don’t stop when I’ve found a match, because, well, there could be another match somewhere. Also, if there is no match of the name between the two files, my command ignores that name, not issuing any output. But, I think that makes sense given what the questioner asks.

So, Tim… does PowerShell have a nifty little built-in or something to make this easier than running through a couple of FOR loops? Inquiring minds what to know.

Tim tags in for Ed:

For loops! We don’t need no stinking For loops!

The first thing to do is import the files. Since there is a space between the columes we can use Import-CSV with a delimiter of the space character. Also, there is no header information so we have to specify it.

PS C:\> Import-Csv length.txt,hash.txt -Delimiter " " -Header File,Data File Data ---- ---- 001.jpg 4227504 002.jpg 4600982 003.jpg 4271719 001.jpg a088531884ee5eb520e98b3e9e18283f29e13d25 002.jpg 77febb1498b2926ee6a988c97f3457e38736456d 003.jpg 922bcb001d025d747c2ee56328811a4270b62079 ...

We have all the data, so now it can be grouped by the file name using Group-Object (alias group).

PS C:\> Import-Csv length.txt,hash.txt -Delimiter " " -Header File,Data | group file

Count Name Group ----- ---- ----- 2 001.jpg {@{File=001.jpg; Data=4227504}, @{File=001.jpg; Data=a088531884ee5eb520e98b3e9e18283f29e13d25}} 2 002.jpg {@{File=002.jpg; Data=4600982}, @{File=002.jpg; Data=77febb1498b2926ee6a988c97f3457e38736456d}} 2 003.jpg {@{File=003.jpg; Data=4271719}, @{File=003.jpg; Data=922bcb001d025d747c2ee56328811a4270b62079}} ...

We have the data grouped like we want, but we still need to massage it a bit so we can get the formate we want.

PS C:\> Import-Csv length.txt,hash.txt -Delimiter " " -Header File,Data | group file | Select @{Name="Hash";Expression={$_.Group[1].Data}}, Name, @{Name="Length";Expression={$_.Group[0].Data}} Hash Name Length ---- ---- ------ a088531884ee5eb520e98b3e9e18283f29e13d25 001.jpg 4227504 77febb1498b2926ee6a988c97f3457e38736456d 002.jpg 4600982 922bcb001d025d747c2ee56328811a4270b62079 003.jpg 4271719 ...

The Select-Object (alias select) cmdlet allows for custom expressions which was used to get the hash and the length. The “Group” object contains multiple items and each can be access by its index value, 0 is the length and 1 is the hash.

Fileless PowerShell

The initial task was to get the file name, length, and hash from separate files and combine them in to one. Let’s try this again without using files.

This would be very easy if powershell just had a hashing cmdlet, but it doesn’t. However, we can do hashing by using the .NET library and some very ugly PowerShell. Maybe in v3 we will get a Get-Hash cmdlet, but it seems as likely as the addition of Get-Unicorn or Get-MillionDollars.

So we need some hash, but not the kind that is illegal in 49 states, we need the hash of a file. Here is how we get it.

PS C:\> PS C:\> gci 001.jpg | % { (New-Object System.Security.Cryptography .SHA1CryptoServiceProvider).ComputeHash($_.OpenRead()) }

We use the SHA1CryptoServiceProvider .NET class, but it adds another bump since it doesn’t take files as input and will only take a stream. It isn’t hard to get the stream though, all we need to use is the OpenRead method of our file object. If that wasn’t enough, there is another problem, the output.

PS C:\> PS C:\> gci 001.jpg | % { (New-Object System.Security.Cryptography .SHA1CryptoServiceProvider).ComputeHash($_.OpenRead()) } 160 136 83 24 ...
The result is an array of bytes. So we have to convert that to hex and combine it together.

PS C:\> gci 001.jpg | % {$hash=""; (New-Object System.Security.Cryptography .SHA1CryptoServiceProvider).ComputeHash($_.OpenRead()) | % { $hash += $_.ToString("X2") }; $hash} a088531884ee5eb520e98b3e9e18283f29e13d25

We use the ToString method with the format string X2 to convert each byte to hex. The X converts it to hex, and the 2 will make sure the output is two characters wide (0A vs A). We then use the variable $hash to stitch our bytes together to get the full hash.

Now let’s see the full command.

PS C:\> gci *.* | select @{Name="Hash";Expression={$hash=""; (New-Object System.Security.Cryptography.SHA1CryptoServiceProvider).ComputeHash($_.OpenRead()) | % { $hash += $_.ToString("X2") }; $hash}}, name, length Hash Name Length ---- ---- ------ a088531884ee5eb520e98b3e9e18283f29e13d25 001.jpg 4227504 77febb1498b2926ee6a988c97f3457e38736456d 002.jpg 4600982 922bcb001d025d747c2ee56328811a4270b62079 003.jpg 4271719 ...

The first thing we do is get all the files in the currect directory using Get-ChildItem (aliased as gci or dir). That is piped in to Select-Object (aliased as select) to get the hash, filename, and size. The Select-Object cmdlet allows us to get properties of the pipeline object as well as creating a custom expression. In our case we will use the custom expression to calculate the hash.

Our results are in object form and can be piped to a file with Out-File or Out-Csv.

So the task is complete, but let’s pretend for a second we had the fictional Get-Hash cmdlet. If we had our leprachaun our command might look something like this:

PS C:\> gci *.* | select @{Name="Hash";Expression={Get-Hash $_ sha1}, name, length

If only getting hash was easier in Windows.

admin | Dec 01, 2009 | Comments 0

Proper Attribution

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Hal starts off:

Back in Episode #54 we got a chance to look at the normal permissions and ownerships in the Unix file system. But we didn’t have room to talk about extended file attributes, and that’s a shame. So I thought I’d jot down a few quick pointers on this subject.

The Linux ext file systems support a variety of additional file attributes over and above the standard read/write/execute permissions on the file. Probably the most well-known attribute is the “immutable” bit that makes a file impossible to delete or modify:

# touch myfile # chattr +i myfile # lsattr myfile ----i-------------- myfile # touch myfile touch: cannot touch `myfile': Permission denied # rm myfile rm: cannot remove `myfile': Operation not permitted # ln myfile foo ln: creating hard link `foo' => `myfile': Operation not permitted # chattr -i myfile # rm myfile
As you can see in the above example, you use the chattr command to set and unset extended attributes on a file, and lsattr to list the attributes that are currently set. Once you set immutable on a file (and you must be root to do this), you cannot modify, remove, or even make a hard link to the immutable file. Once root unsets the immutable bit, the file can be modified or removed as normal. It’s not uncommon to see rootkit installation scripts setting the immutable bit on the trojan binaries they install, just to make them more difficult for novice system administrators to remove.

But there are many other extended attributes that you can set on a file. For example, the append-only (“a”) attribute means that you can add data to a file but not remove data that’s already been written to the file. The synchronous updates attribute (“S”) means that data that’s written to the file should be flushed to disk immediately rather than being buffered for efficiency– it’s like mounting a file system with the “sync” option, but you can apply it to individual files. There’s also a dir sync attribute (“D”) that does the same thing for directories, though it’s really unclear to me why this is a separate attribute from “S”. The data journalling attribute (“j”) is equivalent to the behavior of mounting your ext3 file systems with the “data=journal” option, but can be applied to individual files.

As you can see in the output of lsattr in the example, however, there are lots of other possible extended attribute fields. Many of these apply to functionality that’s not currently implemented in the mainstream Linux kernels, like “c” for compressed files, “u” for “undeletable” files (meaning the file contents are saved when the file is deleted so you can “undo” deletes), “s” for secure delete (overwrite the data blocks with zero before deallocating them), and “t” for tail-merging the final fragments of files to save disk space. Then there are attributes like the “no dump” attribute (“d”) which means the file shouldn’t be backed up when you use the dump command to back up your file systems: “d” isn’t that useful because hardly anybody uses the dump command anymore. There are also a bunch of attributes (E, H, I, X, Z) which can be seen with lsattr but not set with chattr.

So in general, “i” is useful for files you want to be careful not to delete, and “a” and possibly “S” are useful for important log files, but a lot of the other extended attributes are currently waiting for further developments in the Linux kernel. Now let’s see what Ed’s got going for himself in Windows land.

Ed finishes it up:

In Windows, the file and directory attributes we can play with include Hidden (H), System (S), Read-only (R), and Archive (A). H, S, and R are pretty straightforward, and function as their name implies. The Archive attribute is used to mark files that have changed since the last backup (the xcopy and robocopy commands both have a /a option to make them copy only files with the Archive attribute set).

You can see which of these attributes are set for files within a given directory using the well-named attrib command. The closest thing we have to the Linux immutable attribute used by Hal above is the Read-Only attribute, so let’s start by focusing on that one, mimicking what Hal does (always a dangerous plan).

C:\> type nul >> myfile

Note that we don’t have a “touch” command on Windows, so I’m simply appending the contents of the nul file handle (which contains nothing) into myfile. That’ll create the file if it doesn’t exist, kinda like touch. However, it will not alter the last modified or accessed time, unlike touch. Still, it’ll work for what we want to do here.

C:\> attrib +r myfile C:\> attrib myfile A R C:\tmp\myfile

Here, we’ve set the read-only attribute on myfile using the +r option, and then listed its attributes. Note that it had the Archive attribute set by default. We could specify a whole list of attributes to add or subtract in a single attrib command, such as +s -a +h and so on. Note that you cannot add and remove the same attribute (e.g., +r -r is forbidden).

Now, let’s try to see how this attribute is similar to what Hal showed earlier for the immutable stuff:

C:\> type nul >> myfile Access is denied.

C:\> del myfile C:\tmp\myfile Access is denied.
C:\> attrib -r myfile

C:\> del myfile

To remove all attributes, you could run:

C:\> attrib -h -s -r -a [filename]

Beyond attrib, we can also use the dir command to list files with certain attributes, using the /a option. For example, to list all hidden files, we could run:

C:\> dir /b /ah boot.ini Config.Msi IO.SYS MSDOS.SYS NTDETECT.COM ntldr pagefile.sys RECYCLER System Volume Information

I used the /b option here to display the bare form of output so that I omit some clutter.

If you want to see non-hidden files, you could run:

C:\> dir /a-h

You can bundle multiple attributes together as well in dir using a slightly different syntax from the attrib command. With dir, you just smush together all the attributes you want to see, and indicate the ones you don’t want with a minus sign in front of them. For example, if you want to see read-only files that are not hidden but that are also system files, you could run:

C:\> dir /ar-hs

A lot of people get thrown off by the fact that, by default, the dir command omits hidden and system files from its output. Consider:

C:\> type nul > myfile C:\> type nul > myfile2 C:\> type nul > myfile3 C:\> attrib +h myfile C:\> attrib +s myfile2 C:\> attrib +r myfile3

C:\> attrib A H C:\tmp\myfile A S C:\tmp\myfile2 A R C:\tmp\myfile3

C:\> dir /b myfile3
This issue comes up rather often when playing the Capture the Flag game in my SANS 560 class on network penetration testing. Attendees need to grab GnuPG keys from target accounts, with the keys acting as the flags in the game. Rather often, folks get command shell on a target box, change into the appropriate directory for the GnuPG keys, and run the dir command. They see… NOTHING. Inevitably, a hand goes up and I hear “Someone deleted the keys!”. I respond, “You know, GnuPG keys have the hidden attribute set….” The hand goes down, and the happy attendee snags the keys.

You see, the dir command with the /a option lets us specify a set of attributes we want or don’t want. If you want to see all files regardless of their attributes, use dir with the /a option, but don’t include any specific attributes in your list after the /a. That way, you’ll see everything:

C:\> dir /b /a myfile myfile2 myfile3

With this functionality, some people think of the /a option of dir as meaning “all” or “anything”, but it really is more accurate to think of it as “attributes” followed a blank list of attributes. In my own head, when I type “dir /a”, I think of it as “Show me a directory listing with attributes of anything.”

There’s one last thing about attributes I’d like to cover. It’s really annoying that the Windows file explorer doesn’t show system files by default. This is a travesty. Darnit, I need to see those files, and hiding them from me is just plain annoying. I can accept hiding hidden files, because, well, they are hidden. But system files? Who thought of that? It makes Windows into its own rootkit as far as I’m concerned. Because of this, one of the first things I do with any Windows box I plan on using for a while is to tweak this setting. You can change it in the Explorer GUI itself by going to Tools–>Folder Options–>View. Then, deselect “Hide protected operating system files (Recommended)”. Recommended? I guess Microsoft is trying to protect its operating system files from clueless users. Still, this is just plain annoying. Oh, and while you are there, you may as well deselect “Hide extensions for known file types”, which is yet another rootkit-like feature that confounds some users when they are looking for files with specific extensions. And, finally, you may want to just go whole hog (is that kosher?) and select “Show hidden files and folders.” There, now you have a Windows machine that is almost usable!

admin | Sep 15, 2009 | Comments 0

Shares, Mounts and More

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Ed fires the initial salvo:

I’m sure it happens all the time. A perfectly rational command shell warrior sitting in front of a cmd.exe prompt needs to get a list of mounted file systems to see what each drive letter maps to. Our hero starts to type a given command, and then backs off, firing up the explorer GUI (by running explorer.exe, naturally) and checking out the alphabet soup mapping c: to the hard drive, d: to the DVD, f: to his favorite thumb drive, and z: to his Aunt Zelda’s file share (mapped across the VPN, of course). While opting to look at this information in the GUI might be tempting, I think we can all agree that it is tawdry.

So, how can you get this information at the command line? There are a multitude of options for doing so, but I do have my favorite. Before letting the cat out of the bag (ignore the scratching and muffled meows in the background) with my favorite answer, let’s visit some of the possibilities.

To get a list of available local shares, you could run:

c:\> net share

Share name Resource Remark

------------------------------------------------------------------------------- C$ C:\ Default share IPC$ Remote IPC ADMIN$ C:\Windows Remote Admin The command completed successfully.
That’s a fine start, but it won’t show things like your DVD or thumb drives unless you share them. Also, it leaves out any shares you’ve mounted across the network.

Let’s see… we could get some more local stuff, including DVDs and thumb drives, by running wmic:

c:\> wmic volume list brief Capacity DriveType FileSystem FreeSpace Label Name 32210153472 3 NTFS 14548586496 * C:\ 2893314048 5 UDF 0 SANS560V0809 D:\ 16015360000 2 FAT32 8098086912 E:\

That’s pretty cool, and even shows us full capacity and free space. But, it does have that annoying “DriveType” line with only an integer to tell us the kind of file system it is. You can look at a variety of sites for the mapping of these numbers to drive types. However… be warned! There are a couple of different mappings depending on the version of Windows you use. On my Vista box, the mapping is:

0 = Unknown 1 = No Root Directory 2 = Removable 3 = Fixed 4 = Network 5 = CD-ROM 6 = RAM Disk

Other versions of Windows are lacking the “Root Doesn’t Exist” item, and all the numbers (except 0) shift down by one.

Uh… thanks, Windows, but it would be nice to get that info without having to do the cross reference. Plus, we’re still missing mounted network shares from this list. Hmmm….

Well, as we discussed in Episode #42 on listing and dropping SMB sessions, to get the list of mounted shares across the network, you could run:

c:\> net use New connections will be remembered.

Status Local Remote Network

------------------------------------------------------------------------------- OK Z: \\10.10.10.9\c$ Microsoft Windows Network The command completed successfully.

Gee, that’s nice. It shows you the drive letter and what it’s connected to. But, you know, it’s missing the local stuff.

How can we get it all, in an easy-t0-type command and a nice output format? Well, we could rely on the fsutil command:

c:\> fsutil fsinfo drives

Drives: A:\ C:\ D:\ E:\ Z:\

Ahhh… nicer. At least we’ve got them all now. But, you know, having just the letters kinda stinks. What the heck do they actually map to? We could check individual letter mappings by running:

c:\> fsutil fsinfo drivetype z: z: - Remote/Network Drive

But, you know, this is kind of an ugly dead end. I mean, we could write a loop around this to pull out the info we want, but it’s going to be a command that no reasonable person would just type on a whim, plus it’s not going have enough detail for us.

To get what we really want, let’s go back to our good friend wmic, the Windows Management Instrumentation Command line tool. Instead of the “wmic volume” alias we checked out above, we’ll focus on the very useful “wmic logicaldisk” alias:

c:\> wmic logicaldisk list brief DeviceID DriveType FreeSpace ProviderName Size VolumeName A: 2 C: 3 14548656128 32210153472 * D: 5 0 2893314048 SANS560V0809 E: 2 8098086912 16015360000 Z: 4 3144540160 \\10.10.10.9\c$ 4285337600

Ahh… almost there. The DriveType crap still lingers, but this one is promising. We can check out the available attributes for logicaldisk by running:

c:\> wmic logicaldisk get /?
Digging around there, we can see that name, description, and providername (which shows mounted network shares) could be useful. Let’s make a custom query for them:

c:\> wmic logicaldisk get name,description,providername Description Name ProviderName 3 1/2 Inch Floppy Drive A: Local Fixed Disk C: CD-ROM Disc D: Removable Disk E: Network Connection Z: \\10.10.10.9\c$

Soooo close. It kinda stinks having the drive letter in the middle of our output, doncha think? It should start with that. But, a frustrating fact about wmic is that its output columns show up in alphabetical order by attribute name. The “D” in description comes before the “N” in name, so we see the description first. Try reversing the order in which you request the attributes in the get clause, and you will see that they always come out the same way. Bummer…. We could switch those columns around with a FOR loop and some hideous parsing, but no one would ever want to type that command.

But, there is a solution. It turns out that the drive letter is not just stored in the “name” attribute, but is also located in the “caption” attribute. And, my friends, I don’t have to remind you that “C” comes before “D” in the alphabet, do I? So, yes, we can trick Windows into giving us exactly what we want by running:

c:\> wmic logicaldisk get caption,description,providername Caption Description ProviderName A: 3 1/2 Inch Floppy Drive C: Local Fixed Disk D: CD-ROM Disc E: Removable Disk Z: Network Connection \\10.10.10.9\c$

So, there you have it. Reasonable, typable, beautiful. Life is good.

Hal responds:

When Unix folks want to answer the “What’s mounted?” question, most of them reach for the df command first:

# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/elk-root 1008M 656M 302M 69% / tmpfs 1.9G 0 1.9G 0% /lib/init/rw varrun 1.9G 156K 1.9G 1% /var/run varlock 1.9G 0 1.9G 0% /var/lock udev 1.9G 3.0M 1.9G 1% /dev tmpfs 1.9G 324K 1.9G 1% /dev/shm lrm 1.9G 2.4M 1.9G 1% /lib/modules/2.6.27-11-generic/volatile /dev/sda1 236M 60M 165M 27% /boot /dev/mapper/elk-home 130G 99G 25G 81% /home /dev/mapper/elk-usr 7.9G 3.3G 4.2G 44% /usr /dev/mapper/elk-var 4.0G 743M 3.1G 20% /var /dev/scd0 43M 43M 0 100% /media/cdrom0 /dev/sdd1 150G 38G 112G 26% /media/LACIE //server/hal 599G 148G 421G 26% /home/hal/data

Frankly, though, I find that the mount command actually provides a lot more useful data about the mounted file systems than just the amount of available space that df shows:

# mount /dev/mapper/elk-root on / type ext3 (rw,relatime,errors=remount-ro) tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755) /proc on /proc type proc (rw,noexec,nosuid,nodev) sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) varrun on /var/run type tmpfs (rw,nosuid,mode=0755) varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777) udev on /dev type tmpfs (rw,mode=0755) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620) fusectl on /sys/fs/fuse/connections type fusectl (rw) lrm on /lib/modules/2.6.27-11-generic/volatile type tmpfs (rw,mode=755) none on /proc/bus/usb type usbfs (rw,devgid=46,devmode=664) /dev/sda1 on /boot type ext3 (rw,relatime) /dev/mapper/elk-home on /home type ext3 (rw,relatime) /dev/mapper/elk-usr on /usr type ext3 (rw,relatime) /dev/mapper/elk-var on /var type ext3 (rw,relatime) securityfs on /sys/kernel/security type securityfs (rw) none on /proc/fs/vmblock/mountPoint type vmblock (rw) binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev) gvfs-fuse-daemon on /home/hal/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=hal) /dev/scd0 on /media/cdrom0 type iso9660 (ro,nosuid,nodev,utf8,user=hal) /dev/sdd1 on /media/LACIE type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096) //server/hal on /home/hal/data type cifs (rw,mand)

Both commands show you the various physical and logical file systems on the machine, plus information on shares (like //server/hal) and removable media devices (like /dev/scd0). But the extra file system type information (ext3, iso9660, cifs, etc) and mount options data that the mount command provides is typically more useful to auditors and forensic examiners because it provides a better picture of how the devices are actually being used.

The one thing that’s missing from both the df and mount output is information about your swap areas. You need to use the swapon command to get at this information:

# swapon -s Filename Type Size Used Priority /dev/mapper/elk-swap partition 4194296 5488 -1

If you’re running on hardware with a PC BIOS, then your OS probably also includes the fdisk command:

# fdisk -l

Disk /dev/sda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xed1f86f7
Device Boot Start End Blocks Id System /dev/sda1 * 1 31 248976 83 Linux /dev/sda2 32 19457 156039345 83 Linux

Disk /dev/sdd: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xb90dbb65

Device Boot Start End Blocks Id System /dev/sdd1 * 1 19457 156288321 7 HPFS/NTFS
Aside from giving you physical geometry information about how your disks are laid out, fdisk might also show you file systems that are not currently mounted.

Astute readers might have noticed a discrepancy between the output of the mount and fdisk commands. Let me add some command line options to each command to help highlight the difference:

# fdisk -l /dev/sda

# mount -t ext3 /dev/mapper/elk-root on / type ext3 (rw,relatime,errors=remount-ro) /dev/sda1 on /boot type ext3 (rw,relatime) /dev/mapper/elk-home on /home type ext3 (rw,relatime) /dev/mapper/elk-usr on /usr type ext3 (rw,relatime) /dev/mapper/elk-var on /var type ext3 (rw,relatime)

We see in the fdisk output that /dev/sda is split into two partitions, and we can see in the mount output that /dev/sda1 is mounted on /boot. But what are all those /dev/mapper/elk-* devices and how do they map into the apparently unused /dev/sda2 partition?

What you’re seeing here is typical of a system that’s using the Linux Logical Volume Manager (LVM). LVM is a mechanism for creating “soft partitions” that can be resized at will, and it also ties in with a bunch of other functionality, some of which we’ll encounter shortly. Other flavors of Unix will typically have something similar, though the exact implementation may vary. The high-level concept for Linux LVM is that your file systems each live inside of a “logical volume” (LV). A set of logical volumes is a “volume group” (VG), and VGs live inside of a “physical volume” (PV). You can think of the PV as the actual physical partition on disk.

To take an example from the output above, the /home file system lives inside the LV /dev/mapper/elk-home. You can use the lvdisplay and vgdisplay commands to get information about the LV and VG, and these commands would show you that “elk-home” and all the other LVs on the system are part of the VG “elk”. But in order to figure out the mapping between the VG and the PV on disk, you need to use the pvdisplay command:

# pvdisplay --- Physical volume --- PV Name /dev/mapper/sda2_crypt VG Name elk PV Size 148.81 GB / not usable 1.17 MB Allocatable yes (but full) PE Size (KByte) 4096 Total PE 38095 Free PE 0 Allocated PE 38095 PV UUID rOwdB9-r8Io-1IIA-ITRK-TPjE-eF98-RkGqVN

You’ll note that the “PV Name” lists a device name that doesn’t look like a physical partition like /dev/sda2. That’s because in this case my PV is actually an encrypted file system that was created using the Linux disk encryption utilities. That means we have to go through one more level of indirection to get back to the actual physical disk partition info:

# cryptsetup status sda2_crypt /dev/mapper/sda2_crypt is active: cipher: aes-cbc-essiv:sha256 keysize: 256 bits device: /dev/sda2 offset: 2056 sectors size: 312076634 sectors mode: read/write

Whew! So let’s recap. /home is an ext3 file system inside of /dev/mapper/elk-home, which we learn from the mount command. lvdisplay would tell us which VG this volume was part of, and vgdisplay would give us more details about the “elk” VG itself. pvdisplay normally gives us the mappings between the VGs and the physical partitions, but in this case our PV is actually a logical volume inside of an encrypted file system. So we need to use cryptsetup to dump the information about the encrypted volume, including the actual physical device name. That’s a lot of layers, but it’s really not that awful to deal with in practice.

admin | Sep 08, 2009 | Comments 0

Download and unzip a file to multiple directories.

From Nukeit.org

This script was written because some plugins used on more than one blog get updated every day, yet don’t have a public source control. To use, create a file called dirfile.txt in the same directory as you save the script. Add the full paths to each of your target directories, and save. Run the script and paste in the url to the zip you want to download and extract. That’s it!

Download & unzip a file to multiple directories. (59)

admin | Jun 25, 2009 | Comments 0

File Linking

From: COMMAND LINE KUNG FU: PaulDotCom, Ed Skoudis, Hal Pomeranz, byte_bucket

Paul pops off:

Creating links between files is a handy feature in UNIX/Linux systems. There are many instances where you need to have a copy of the file (or dirctory) in a particular location, but only want to maintain one original. For example, I was running a program to check the security of my Apache configuration file. It expected the file to exist in “/usr/local/apache2/conf/httpd.conf”, but the original file was located at “/etc/httpd/conf/httpd.conf”. To solve this problem I created a “soft” link as follows:

$ ln -s /etc/httpd/conf/httpd.conf /usr/local/apache2/conf/httpd.conf

The above “ln” command takes the “-s” flag to indicate a soft link, which creates a pointer to the original file. Next you specify the original file, followed by the file that will point to the original. Many will forget which one comes first (the original or the pointer), so don’t forget that the original file always comes first :) Oh, and you can view the links by using the ls -l command:

$ ls -l /usr/local/apache2/conf/httpd.conf lrwxrwxrwx 1 root root 26 Apr 21 13:57 /usr/local/apache2/conf/httpd.conf -> /etc/httpd/conf/httpd.conf

Hal chimes in:

Let me show you one more useful trick with the “ln” command. You can actually create symlinks to an entire directory of files with a single “ln” command:

# cd /usr/local/bin # ln -s ../depot/clamav/current/bin/* .

First we “cd” to /usr/local/bin. The “ln” command creates a link to every object under /usr/local/depot/clamav/current/bin. The names of the links in /usr/local/bin will have the same name as the files under …/clamav/current/bin.

This is how I manage software that I’ve built from source on my systems. In fact, …/clamav/current is itself a symlink to a directory like …/clamav/0.95.1. Whenever I build the latest version of ClamAV, I install it in its own …/clamav/ directory and just change the …/clamav/current symlink to point to the latest and greatest version. Since all the symlinks under /usr/local/{bin,sbin,etc,lib,include} are expressed using the …/clamav/current link, every other link in the hierarchy automatically starts pointing at the right version as soon as I change the …/clamav/current link. And it’s easy to revert too, just in case the new version isn’t working for some reason. Slick.

Ed responds:

Sadly, Microsoft never got around to implementing a pure-play shortcut-creating feature inside of cmd.exe. Because of that, several folks have released third-party tools that do so. Some nice ones include the NT resource kit tool simply called shortcut.exe, Pixelab’s xxcopy, and NirSoft’s NirCmd.

But, downloading a third-party tool isn’t our way at this here blog. So, we must explore other options.

While cmd.exe itself doesn’t have a feature for creating shortcuts, wscript, which is built in, does. There are many examples out on the Internet for creating shortcuts with wscript, but I’ve boiled them down to their bare minimum:

set WshShell = WScript.CreateObject("WScript.Shell" ) set oShellLink = WshShell.CreateShortcut( Wscript.Arguments.Named("shortcut") & ".lnk" ) oShellLink.TargetPath = Wscript.Arguments.Named("target") oShellLink.Save

The above script takes two arguments: the name of the target you want to create a shortcut to (/target:) and the shortcut name itself (/shortcut:). Note that the target could be a file or a directory. To create a shortcut using this script, we could dump all of that stuff above into a file called shortcutter.vbs, and then run it with the wscript interpreter.

“Ah… but that would be a scripting solution and not a command line,” you might say. “You need to create a single command line that addresses the challenge.”

Thanks for the delightful reminder. What, are you on Hal’s payroll? Don’t you have anything better to do with your time than taunt me? ;)

OK… I’ll take your input and respond with this for a command line:

C:\> echo set WshShell = WScript.CreateObject("WScript.Shell" ) > shortcutter.vbs & echo set oShellLink = WshShell.CreateShortcut( Wscript.Arguments.Named("shortcut") ^& ".lnk" ) >> shortcutter.vbs & echo oShellLink.TargetPath = Wscript.Arguments.Named("target") >> shortcutter.vbs & echo oShellLink.Save >> shortcutter.vbs & wscript shortcutter.vbs /target:[source] /shortcut:[shortcut]
It pretty much types itself, doesn’t it? Easy!

Uh…. or not.

I’m simply creating the vbs script, which I’m naming shortcutter.vbs, and then invoking it to create the shortcut. I don’t delete it at the end, because I want to keep it around for future uses. These things come in handy, you know.

admin | May 15, 2009 | Comments 0

Securely Destroy Data

From: http://www.commandlinefu.com

$ shred targetfile

GNU shred is provided by the core utils package on most Linux distribution (meaning, you probably have it installed already), and is capable of wiping a device to DoD standards.

You can give shred any file to destroy, be it your shell history or a block device file (/dev/hdX, for IDE hard drive X, for example). Shred will overwrite the target 25 times by default, but 3 is enough to prevent most recovery, and 7 passes is enough for the US Department of Defense. Use the -n flag to specify the number of passes, and man shred for even more secure erasing fun.

Note that shredding your shell history may not be terribly effective on devices with journaling filesystems, RAID copies or snapshot copies, but if you’re wiping a single disk, none of that is a concern. Obviously shredding a disk would take quite some time.

admin | Apr 28, 2009 | Comments 0