Posted by Adam Ruth on Fri, Jul 30, 2010
I've been recently catching up on watching Red vs. Blue which is a web video series created by a company called Rooster Teeth. If you're already familiar with the show, then you can skip all of this. Otherwise, you're in for a treat.
Red vs. Blue is a comedy series set in the world of Microsoft's Halo video game. The footage is all from within the game itself, with conversations between characters as they get into various predicaments. The basic premise of the show is a satirical look at these types of first-person shooter games. The Red and Blue armies have outputs located in an isolated box canyon called Blood Gulch, which is just the kind of map that you might find yourself in while playing the game. No real reason for being there, other than to set up a battle.
The show's greatest strength is its characters, who are a collection of clueless misfits bumbling around their artificial world finding new and interesting ways to irritate each other. Plenty of nerd based humor makes its way into the dialog and situations, which is funny even if you've never played a computer game. Each episode is a short couple of minutes, and you can watch most of them on the RoosterTeeth YouTube channel.
Enjoy!
Follow me on Twitter @AdamRuth
Posted by Adam Ruth on Wed, Jul 28, 2010
In a previous post I went over three cmdlets for formatting data. Each of those cmdlets let you select which object properties you want to show. You aren't just limited to the existing properties on the objects, though, you can use Calculated Properties which let you format the properties in any way you like. The syntax for calculated properties looks a little odd at first, but it's actually quite simple.
For example, this expression displays the time between the creation and last write times of files:
There are a few things here that need explaining:
- The back-tick at the end of the first line allows you to type in commands on multiple lines, making it easier to read and write long commands.
- Calculated properties are defined as Hash Tables, which are simple name/value containers. They are created by putting the names and values inside of @{} separated by semicolons.
- Calculated properties require that the hash table has two values: Name and Expression.
- The Expression value in the hash table is itself enclosed in curly braces {} which is the way you pass PowerShell script code as a property to a cmdlet.
- The value $_ inside of the Expression is the object that is being formatted. In the above example it is referenced twice to pull out the two properties CreationTime and LastWriteTime.
- CreationTime and LastWriteTime are .NET objects called DateTime and they have a number of properties and actions themselves. The example uses the Subtract method which gives the difference between two DateTime objects.
- The object returned from the Subtract method is called a TimeSpan. It displays as days.hours:minutes:seconds.milliseconds. It also has a number of properties to get these values individually.
As you can see, showing a TimeSpan like that is not the easiest to read. We can display just the days like this:
One PowerShell construct that is very helpful when formatting output is the -f operator. This takes text with embedded placeholders in it and replaces them with formatted values. A simple example:
There are two placeholders which insert the two values (placeholder numbering starts with 0.) Not only can the -f operator insert the given values, but it can also format numbers in a variety of ways. Consider this example:
This example shows the size of each file in megabytes, formatted out to two decimal places. It starts by dividing the Length of the file by 1Mb (which is a built-in PowerShell value handy when dealing with file sizes along with 1kb, 1gb, and 1tb.) This value is then passed to a format placeholder {0:0.00}. The text after the colon defines how the number is to be displayed. In this case, it will show one at least one digit to the left and no more than 2 digits to the right of the decimal. There are many different types of formatting rules you can use, here's Microsoft's documentation for more details.
Once you understand the basics of Calculated Properties, there really isn't any limit to how values can be formatted. Calculated properties aren't just for making output easy to read, but they come in very handy when you need to pass the output of some command to a program that expects things to be in a specific format. Consider using them any time you have to massage data manually.
Need help with PowerShell? Post a question in our new PowerShell forum and get an answer quick.
Posted by Adam Ruth on Mon, Jul 26, 2010
Photo by Tom Anderson
The Internet is full of successful protocols such as HTTP, FTP, DNS, SNMP, IMAP and myriad others. Let us not forgot those protocols that never quite reached stardom, or died right out of the gate. Here are the top 10 protocols that, for one reason or another, never made it mainstream.
10. HTTPf
Hyper Text Transfer Protocol fecure
Similar to HTTPS but everything is encrypted by converting to old English.
9. FCP
File Corruption Protocol
Never really found a practical use.
8. PIDP
Pornography Instant Delivery Protocol
Renamed HTTP.
7. CBDP
Caffeinated Beverage Delivery Protocol
Rejected after computer modelling showed it would require 132% of the world's aluminium resources to make the cans.
6. BFMP
Butter Finger Messaging Protocol
Problems with dropped packets spelled its demise.
5. VOAT
Voice Over A Telephone
Duplicated talking on the phone.
4. N411SS
Nigerian 411 Scam Service
A member of Nigerian royalty is still trying to get this one started, if he could only get some seed money.
3. RFTP
Renewable File Transfer Protocol
No different than regular FTP, but used exclusively by Hybrid owners.
2. PLPR
Priceline Packet Routing
William Shatner allows packets to name their own number of hops.
1. SJRDIP
Steve Jobs Reality Distortion Interface Protocol
Catastrophically fails if you touch the antenna in the wrong place.
There are many I've missed, so please share them in the comments.
Follow me on Twitter @AdamRuth
Posted by Adam Ruth on Fri, Jul 23, 2010
In PowerShell everything is a .NET object, each with a variety of different properties. You'll notice that most of the time the output you see from basic commands only gives you a small subset of what's available. This is because PowerShell has built-in rules for how and what to display for most of the standard objects. Usually this is done so that you normally only see what's most important and aren't overwhelmed with too much data. For example, the dir command returns a set of FileSystemInfo objects but only shows you a few of the 20+ properties that are available.
You can override the default output for any object and PowerShell provides a set of cmdlets for just this purpose: Format-Wide, Format-Table, Format-List, and Format-Custom. I'll cover Format-Custom in another post, so for now here's a brief description of the first 3.
Format-Wide
This cmdlet simple takes one property of each object and writes them out across the window. This is similar to the /w option when using dir in cmd.exe.
Notice that to use the cmdlet you need to pipe the output of the dir command to Format-Wide. Unlike the pipe command in cmd.exe, you aren't just sending the text from the first command but actual .NET objects which allows the second command to have full control when dealing with them.
With Format-Wide you can select the property to show with the -Property parameter. You can also adjust the width of the columns with -Column.
Format-Table
This cmdlet formats objects just like the default dir command, but you have control over what properties show up and how they are formatted. For example:
In order to adjust the widths of the columns independently you need to create a View and pass it to the cmdlet, but that's a topic for its own posting. You can even group the output by unique properties of the files.
Format-List
This cmdlet formats the output as a list of properties. It's similar to the table output, but rotated 90 degrees.
Just like Format-Table you can decide which properties to show. You can use wildcards to select multiple properties (this also works with Format-Table.)
If you just use * for the property list you'll see all of the properties of the object. This is handy while exploring objects to see what information you can work with.
Hopefully this quick overview will get you exploring the different ways to output data in PowerShell. In future posts I'll discuss more advanced topics such as Format-Custom, Views, and Calculated Properties.
Follow me on Twitter @AdamRuth.
Posted by Shawn Anderson on Wed, Jul 21, 2010
Photo by HikingKid
I was working on a problem last week that caused me to, more than once, blaspheme various deities known to humankind. I had 10 servers each running SQL 2008 Express. An application that was supposed to process data in these databases had unexpectedly quit and, thus, I had 4 tables in each DB that had more than 60,000 rows of data to process and more data coming in. The application which had to process the data was using 2 GB of memory and quickly rising. 2 GB may not seem like a lot but we are talking an x86 server with 4 GB of RAM.
The thing about all this data, however, is that it is largely time specific. Online status, computer logons (not user logons) etc. After 30 minutes or so, the data was not important any longer (as it had been replaced by other logon events and online status updates.)
The amount of data that had to be processed at each of the servers was so immense that the servers were not responding the way I needed them to. Since I didn't need the data, I simply decided to purge the outdated data from certain tables.
I opened up Admin Arsenal and selected and ran a Remote Command against a collection which held all my servers on which Microsoft SQL Express was installed.
Since Admin Arsenal's Remote Command feature will actually run the command on each selected system there was no need for me to provide a server name.
Within 2 minutes all of my databases were processing current data and all was well.
I needed a quick solution and I got one. I couldn't log on to each system from the SQL Database Manager application provided by Microsoft. The overhead of each connection was so great that even increasing the timeout to 300 seconds was not enough. Plus, even if I COULD have attached to each DB via the Manager I would have been working against myself. I needed all the unneccesary data gone from all the systems fast. Individual attention to each would take too long.
Remember, sometimes your solution is just one command away...
Follow me on Twitter @ShaneCorellian
Posted by Adam Ruth on Mon, Jul 19, 2010
Photo by kretyen
This article on Tech Republic showing a computer that was the victim of a lightening strike got me thinking. (WARNING: if you're squeamish about graphic computer carnage, don't view the photos.) What it got me thinking about was static and computers. Back when I first got into IT I constantly heard about the dangers of static electricity. It was only the most daring soul who would breach the hermetic seal of the computer case without guaranteeing they were properly grounded.
It didn't take me long to start thinking that there was more hype than substance to the claims of static damaging computer components. I have no doubt that are plenty of components lying in pauper's graves that were killed by an errant arc from someone's fingertip. But how many times was static the go-to scapegoat when something went wrong that no one wanted to bother explaining? I remember very early in my career when I accidentally dropped a screw onto the motherboard of a running computer, so I am certain why that ISA slot was broken (I did say it was early in my career.) However, there was a time when the mysterious death of a component almost required that static be the first straw grasped.
It hardly seems that way anymore. "You must have static-ized it," is very rare these days. Have we System Administrators gotten better at dealing with static or do we know better that static is usually just an easy excuse? I'm sure it's still possible to build up enough of a charge to fry some modern components, but how likely is it while standing on a tiled floor after touching metal racks and server casings? I'm sure that if your server room is located in the middle of a cotton-ball factory carpeted with 1970s era shag and your company uniform is a corduroy jumpsuit with fuzzy bunny slippers you may want to slip on a grounding strap. Otherwise, we've learned to be less paranoid.
So what does that tell us about modern computing concerns? Are there things today that are like the static of yesteryear where blame seems to automatically go in a knee-jerk fashion? It seems to be when some new, and not yet fully ripe, technology starts to get adopted it continues to be blamed for problems long after those problems have been ironed out. That makes sense, because there was a time when the problems were real, so the knee-jerk reaction was probably right.
Windows instability seems like one of those historical issues. There was a time when a BSOD was almost certainly a bug in the OS and a reboot was the fix. Nowadays, though, a BSOD is more likely to be a hardware problem that isn't going to go away with a simple reboot. I'm sure you can think of some others.
Follow me on Twitter @AdamRuth
Posted by Adam Ruth on Fri, Jul 16, 2010
Photo by dhepnar
I've been following the story of IPv6 for a while now, and I've been following it like I believe most system administrators have been. Which is the standard waiting-for-when-I-have-to-drop-everything-and-get-it-implemented way. I'm sure that this attitude makes some people unhappy, after-all we're running out of IPv4 addresses and I'm keeping us from solving the problem. If we all took the time now to get things switched over then we could avoid IP Addressageddon.
Well, it's not as bad as all that. In my, admittedly limited, experience I have learned a few things which makes me a little less worried. Less worried, but still a bit worried.
1. The death of IPv4 has been greatly exaggerated.
Available IPv4 addresses were supposed to run out this year, but that date has been pushed back to 2011 despite a (somewhat) mad rush on addresses. The date will probably be pushed back again, but each time it does, the cost of not transitioning will increase. No one wants to be left without a chair when the music stops, but on the flip side, no one wants to spend time or money now when there still seem to be quite a few chairs left.
Personally, I think that IPv4 will be with us long until after the IPv6 transition is essentially complete and that some form of technology will keep it alive in the same way that there are still people on dial-up. In the same vein, though, there are good reasons to get off of dial-up.
2. Vendors are already there, so there's no excuse!
One thing that I hear from IPv6 advocates (zealots?) is that vendors are supporting IPv6 in droves and so it should be simple to move our networks over. IPv6 has been in Windows, Linux, various UNIX, and OS X for years (with hardware for about as long) what's the hold-up?
Well, implementing IPv6 at the vendor level is (not to put too fine a point on it) a helluva lot easier than at the system administrator level. It's nice having all of the pieces of the puzzle, but it's the job of putting them together that is the hard work.
3. IPv4 mapping could have been better.
I think that the IPv6 standards committee dropped the ball here. I get the feeling (and this is based solely on my impression, I haven't done enough research to verify) that IPv6 was designed to be as minimally compatible with IPv4 as possible. The idea seems to have been that by making it harder to move to v6 piecemeal it would make the transition happen faster. If that is the case, then I think that the opposite is the result. By increasing the cost of a slow rollout of v6 it made it more attractive to stick with v4 as long as possible.
I may be completely wrong on this, I will admit. There may be technical reasons why a move to v6 couldn't have been very smooth. But I'm thinking of NAT here. When NAT came out it was a godsend for many reasons, even though it was a kludge that has a whole new set of problems. It hit where it needed to, by allowing new nodes to be added with very little disruption to existing infrastructure. IPv4 mapping seems to be trying to provide a seamless move to v6 but either it's not as good as it could be or it has really bad PR.
4. IPv6 readability.
IPv6 addresses are hard to read. There, I said it, you can now come to confiscate my nerd credentials.
Really, though, I understand that v6 addresses are 4 times longer than v4 addresses and there's no way that they can be as easy to remember and work with. But the 4 octet scheme of v4 is so ingrained in sys admin culture that it seems crazy to depart from it so much with v6. Sure, eventually we'll all be thinking in v6 subnets and addresses but it's a big jump. I can't help but think that the v6 representation came out of academia instead of from the trenches.
5. Subnetting is still important.
When the move to v6 finally comes, don't give up all of the knowledge you've built up around subnetting. Sure, even the smallest v6 address allocation provides for 65536 subnets each with billions and billions of nodes. But that only takes care of part of the issues with subnetting today. It's still necessary, although easier, to understand where to divide subnets relative to physical routing. It'll be much easier, but not automatic.
Summary
IPv6 is inevitable, so let's all keep an eye on the bouncing ball. But don't panic just yet, the tipping point is still a ways away. It would be nice if the move to v6 was as easy as rolling out a new OS (even though that certainly isn't easy, but it is easier.) But it isn't, so we sys admins will continue to do what we always have done: Make it all work somehow.
Follow me on Twitter
Posted by Adam Ruth on Mon, Jul 12, 2010
One feature of PowerShell that I had a hard time figuring out was how to include one script in another. I have a number of build scripts which build the installers for our products, and there are a lot of steps that are the same between them. Up until recently I was using the proven method of copy/paste to share the functionality, but I finally decided to sit down and take the time to build out my build system properly.
The first thing I needed to do was to put similar functionality into a library and load it into each product's build script. This library consists of a bunch of functions such as incrementing version numbers, building help files, compiling source code, and digitally signing files. The problem is how to get the functions to be available to other scripts. As you may have noticed, when you run a script none of the functions are available to you after the script ends. Here's me running a script that has a single function called Hello:
In order to make the function in the script last after the script ends, it's necessary to run the script preceded by a period:
With that, all symbols in the script persist and you can create any library scripts you may need.
Once you get to the point where you have enough scripts to make modularity necessary, you are to the point you should consider using some source control (for a primer read my whitepaper on the topic.)
Have any other PowerShell tips that may not be obvious to everyone? Post them in the comments and share with the world (or the tiny portion of it that reads this blog, anyway.)
Posted by Adam Ruth on Fri, Jul 09, 2010
It's a question that has been on the minds of system administrators for a long time. As the number of laptops (and now slate devices and even phones) continues to grow and displace desktops, so does the management headache. At some point, it seems, that desktops themselves will become extinct leaving only servers and portable devices left to manage.
Farhad Manjoo proclaims the imminent death of desktops at Slate, he makes some very good points. Eric Raymond makes some good counter-points about how peripherals (especially screens and keyboards/input devices) will always keep people tied to a desk. I think that they may be talking past each other, at least from the perspective of IT folks.
From the standpoint of managing computers, there isn't much (if any) difference between a truly portable computer and one that can be moved between office and home. It's the fact that the computer is used in different environments and at different times that creates the management friction. Even if a user lugged their big, boxy desktop computer between home and work (I've known people to actually do this) it's still a greater burden on system management than a truly portable computer that never leaves the office.
So, while the death of the desktop PC may be exaggerated in the sense that there will continue to be a "desk" involved most of the time, we are soon to see computers moving from desk to desk as the standard and the static workstation becoming the specialized, niche product.
Also, I'd like to add that I can envision a world where screens and keyboards will become unnecessary. I can see wearable computers become more and more practical over time. For those that scoff, I remind them that people scoffed at the ideas of laptops after seeing the first portable computers. I would love to be able to look up IMDB to win bets without even moving out of my hammock or so much as picking up anything. That's when my Homer-Simpson-esque dreams can finally come true.
Posted by Shane Corellian on Wed, Jul 07, 2010
The fun side of studying
Part III of my "Understanding or Memorizing" series. In 2006 a friend and former co-worker came into our offices and saw Adam reading about some new Systems Management technologies that were in the early phases of acceptance. Our mutual friend went up to Adam and, interrupting his studies, said "Why are you studying? I thought you were a wizard!" I still love Adam's reply (paraphrased) "If I am a Wizard then it's only because I actually do study."
Touché. (By the way, Adam IS a wizard. Ask anyone who has ever worked with him...that dude is siiiick.)
Keep sharp. Be constantly learning about new and evolving technology, especially if it has to do with your career. Do you know where to go when you have questions? Have you signed up with the SysAdmin Network? What about ServerFault and The Nubby Admin?
I live and die by O'Reilly books. Mine are dog-eared, written on, highlighted, and some pages have tea or coffee stains.
Eric Sink wrote a fantastic book called "Eric Sink On The Business Of Software". This was one of the best books that I read in 2007. On Page 108 he gives some recaps related to hiring qualified developers (easily translated to Sys Admins). The first three points are of special interest to me:
- Can this candidate bring something to the team that nobody else has?
- Is this candidate constantly learning?
- Is this candidate aware of his/her weaknesses and comfortable discussing them?
I love being around people who are self-aware. Know your weaknesses as well as your strengths! One of the best statements made by Mr. Sink is found on page 125. "...Worrying about how others perceive your [aptitude/intelligence] is a waste of time. The key to a great career is to focus on [learning]..."
Learn or burn.
Follow me on Twitter @ShaneCorellian
Posted by Adam Ruth on Mon, Jul 05, 2010
Photo by mikebaird
The ongoing battle between Google and Microsoft recently heated up with a Google employee going public with details of an unpatched exploit in Windows XP only 5 days after notifying Microsoft of the vulnerability. There appears to be some politics (for want of a nicer word) going on behind the scenes of this episode, and there's plenty of analysis of that out in the blogosphere. What interests me, though, is how this highlights the difficulty that patching poses for the system administrator.
Patches are unavoidable, unfortunately, and there is a conflict between the need to get a patch out quickly and the need to ensure that patches are fully tested before they break things. Vendors cannot test patches with every application out there, especially in-house software, and so there will always be a percentage of patches that break things. This conflict is one of the reasons that Microsoft started Patch Tuesday. They have received some criticism for no longer issuing patches as they become available, but I think that it's actually a fairly good compromise. Large sites need a more predictable release cycle to deal with the flood of patches that have become a security reality. It would be nice of Microsoft could also deliver patches more quickly to smaller customers, but once a patch is released an exploit can be reverse engineered out of it leaving the others exposed. Once a month seems to work pretty well, all-in-all.
Now administrators have gotten in the cross-fire in what appears to be a battle between two behemoths. Dealing with patches is hard enough as it is, it's unfortunate that this eruption could probably make things worse. Would we have been worse off had Microsoft dragged its feet in releasing a patch (which is what Google seems to claim would have happened had they not gone public) or could sites be affected in the time before the patch comes out? I honestly don't know, but I tend to lean towards Google's actions making things worse. Luckily these types of public disclosures don't happen very often, so I really hope we're not seeing the start of a trend.
In the end it still comes down to the lowly system administrator. Patches need to be tested and deployed, no way to get around it. Hopefully these kinds of issues can lead to better patching processes, but in the mean time it's us admins that really on the line.
Deploying patches? Read our white paper on software deployment.
Posted by Shawn Anderson on Fri, Jul 02, 2010
A couple of weeks ago I took part in a global chat hosted by Jason Calacanis on his This Week in Startups weekly podcast.
Here is a snippet of our discussion, which starts around minute 55:00.
Jason: In the enterprise are people moving away from Windows, are Apple and Macs starting to get some foothold or is that something that MacHeads like to think?
Shawn: Not in the Enterprise. To us the entperprise is thousands of computers. In the small-to-medium (SMB) business... well not even there. In some of the niche markets like art studios you'll see a lot of Macs. [I then proudly display my MacBook Pro to the camera]. I use a Mac for personal but I make money on Windows [because] that's just what companies are using. You've heard of a big Linux swelling but you don't really see it in the marketplace. So we definitely focus on Windows.
Jason: So what do you think in the next five, ten years, do you think you'll see Apple starting to take market share in the enterprise, do you think you'll see Linux taking marketshare in the enterprise?
Shawn: Linux has already started to slow down, I think you'll still see them make a good solid hold on servers, but they've already lost on the desktop applications. As far as Mac, yeah, I think if you can integrate them [with Windows environments] a little better - the move to Intel was a step in the right direction, but as far as five to ten years I think you'll see more Software as a Service (SaaS) and I think you'll see a lot more virtualization.
I've reflected on my answer to Jason and I want to expand somewhat. Apple makes computers that consumers want. They like Apple products because they look so good, are easy to use, and in general do a great job of getting out the way of you and your apps.
But these benefits come with a jump in cost. This jump, as well as the cost of managing Windows & Mac's simultaneously, are no doubt what the CTO and CFO's of corporate America take into consideration. When it's time to puchase 300 new laptops, CFO's aren't overheard saying "I'm looking for an easy to use computer that is a beautiful as the Grand Canyon." More likely they're simply saying "get me the cheapest thing that our users already know how to use."
This introduces a dichotomy. - Apple makes billions skating to where the puck will be, yet virtualization is a tiny blip on their radar (Parallels and Fusion). Yet all the experts say that Cloud computing and virutalization are the wave of the future. So who's right?
Both parties. As much advancement as the future will bring, computing will still very much be broken down into consumer and business.
Apple will likely remain a predominiately consumer electronics outfit. Dell and HP will no doubt continue on the slow course of evolution with business computing, churning out powerful desktops and laptops and trying to get a toehold on the personal and mobile computing market. They will probably not make a huge dent on the latter, but will likely rule the roost on the former.
The interesting (authoritative) statistic that I want to see is the number of PC users at work who make the switch to Apple or other leading mobile computing devices for personal use.
Count me in that stat. I used Apple through the early 90's but switched when Windows 95 was released. I gladly jumped back into the Mac camp when they released their first Intel MacBook Pro line. Right now I'm happy camper.
As far as mobile computing goes; I like my iPad but it's more of an ultimate bathroom reader and killer movie player for long plane rides.
Three words: Google Apps Integration.
Follow me on Twitter @ShawnAnderson
Follow Jason Calacanis on Twitter @Jason