Photo by Striatic
"Developers should consider all data input by a user as harmful until proven otherwise." ~ Rocky Heckman
I liked this statement so much I had to use it for this article. Though somewhat unrelated, the sentiment is precisely what I was looking for.
ZDNet Australia ran a story by Josh Taylor who discussed a TechNet 2010 discussion by Rocky heckman about hackers inadvertently sending their malicious code to Microsoft during the development phase of their soon-to-be virus. Heckman went on to say that the same practices have been occuring for over six years, meaning that developers aren't listening to their security counterparts.
Thanks Mr. Heckman. Now, let's expand that statement not to just developers who might have to deal with an outsiders maclicious code, but to our jobs as Windows administrators, specificially as it relates to remotely installing applications to our users desktops.
Just like a realtor cares about location, location, location, sys admins should care about testing, testing, testing.
From time to time we receive suppport requests from users who have pushed the lastest bleeding edge package (or patch) to all of their systems. This is careless and can cause some serious problems.
First off, we define serious problems as anything that:
- prevents or hinders a users ability to complete their tasks
- compromises system security
Testing is a fairly straight forward proposition. Get the application or patch into a controlled environment and test the installation. Once you get an installation that is successful (i.e. the patch or application is working as advertised), review the event logs and test any other critical applications. (Testing critical apps, even though they may be unrelated to the installed app or patch is very important when it comes to installing anything on servers.)
We are big proponents of VMWare, especially the ESXi flavor. We have several in our lab and we use them extensively for testing and support.
Read the documentation of the app or patch. Does it change your security posture in any way? Does it open new ports? Does it necessitate the creation of any service level accounts? Does it make any changes to system files? All of these answers (and more) are included in the documentation that we all just love to read.
I know, I know. Documentation is a bore. But it's part of our jobs as sys admins. Reading docs is like shoveling dirt. It's boring and tedious, but once in a while you'll find a nugget, and that will make the work worthwhile.
I'm a big believer that we'll not as a society achieve true utopian peace until every entity (that is person and business) backs up their data and verifies that those backups actually work. (That's another way of saying I think that utopian thinking is nonsense.) True too the testing of apps prior to major deployments.
Be that as it may, just because there are groups of admins who cowboy up and deploy with reckless abandon doesn't mean that you have to. Just like the guys being chased by the bear realize that they don't have to outrun the bear, they just need to outrun their slowest group member.
Remember, ESXi is free and is only a download away.
Follow me on Twitter @ShawnAnderson
Deploy software for free with PDQ Deploy.
Happy Thanksgiving to our American readers, and for our non-American readers Happy Late November!
I was trying to find some cool nerdy humor for Thanksgiving when I finally gave up and fell back onto my old standby, Cracked.com. So, without further ado: 25 Hand Print Art Projects Way More Badass Than a Turkey.
Enjoy your holiday and/or normal day!
This is a phrase that you never want to hear when discussing a completed remote software deployment project.
Actually, due to its horrific grammar, this is a phrase that you never want to hear under ANY circumstance. Oh, and Yes, this is an actual quote. Not third person. Not from a friend of a friend. I was there. I heard it. My ears will never recover.
No, the quote isn't from Deadwood or any other show which depicts uneducated ignoramuses getting by in a cruel world.
Yes, that quote was spoken by a man in his mid thirties who happened to be two months away from getting his grubby hands on his Master's Degree (from a reputable university, no less).
It was only after missing a freeway exit outside of King Of Prussia, Pennsylvania that prompted my, then, co-worker to utter the words "Was that where we was supposed to went?"
About 18 months ago I wrote about how happy I was with Dameware Mini Remote Control. While I am still very happy with the product I did break a deployment rule that I know well. Basically I screwed up.
What did I screw up? I will sum it up with another quote (this one is actually famous): "Plan your dive and dive your plan." I learned this when I was getting certified for SCUBA. It basically means that we need to plan what we intend to do and then do what we actually planned.
I didn't actually do the due diligence that I should have when helping a customer with rolling out DameWare and as a result I made the management of the application a little more labor intensive. You see, in DameWare Mini Remote Control there are two methods for storing Remote Control configurations on target systems. The default and, apparently, most common method is via the file DWRCS.INI located in the %WINDIR%\System32 directory. The other method which I should have pursued is to store all the configuration data in the Windows registry.
This decision was especially bad because I already have a method for collecting the values of certain registry keys from target systems and storing the data in an Inventory database. I could have, with a simple SQL query, been able to view ALL the DameWare remote control configurations for ALL of the customer's computers.
The fix is not as bad as you may think, but it does require that I rebuild the DameWare Mini Remote Control agent (for future deployments) and make file modifications to all of the computers that currently have the previously deployed agent.
I can do this quietly so as to not disturb the end users but I despise doing anything to production computers that isn't necessary. Deploying the agent with the correct* configuration in the first place would have been, ahem, much better. Needless to say, I am ready for the Windows Admin Drinking Game.
OK, so I did "dive my plan", but diving a crappy plan still sucks. I should have taken more time with my decision. I should have anticipated that modifications to the configuration were inevitable. While we can't necessarily anticipate WHICH configurations will be changed in any project we should know that SOME configurations will most assuredly be modified. Planning an efficient modification process is integral to any successful deployment.
So, when I fully realized my error I found myself muttering "Was that where we was supposed to went?"
* - Correct configuration depends on time, money, quality and blood alcohol content.
This is a repost from Dec. 30, 2009 blog article.
Follow me on Twitter @ShaneCorellian
In the words (OK, word) of Gob Bluth: "C'mon!"
Remember that AA Console (formerly Admin Arsenal) has the awesome ability to run your favorite scripts or IT utilities right from the Tools menu. We call this feature "Custom Tools".
A Custom Tool is a command that exists on the AA console machine. When the Custom tool is selected (either from the Tools menu or a keyboard shortcut that you assign) the command is executed along with any respective command line arguments.
Want to be able to automatically go to the C$ of a target computer? Go to your AA Console Preferences and, in the Custom Tools pane, add the following line:
Open C$ Share=explorer.exe "\\%TARGET%\C$"
The syntax for a custom tool line is
Name [;keyboard shortcut]=command [ARGS]
AA Console will store the computer name in the %TARGET% variable.
If you use DameWare Mini Remote Control, you can initiate a Remote Control session from within AA Console by adding a custom tool entry like this:
DameWare Remote Control;CTRL+ALT+Z="C:\Program Files (x86)\DameWare Development\DameWare Mini Remote Control\dwrcc.exe" -m:%TARGET% -a:1
See additonal arguments that can be passed to DameWare Mini Remote Control.
Would you like to automatically connect to a network registry? Feel free to download one of our free utilities called StartReg.exe. Place this file on your AA Console machine and add the following line to your custom tools:
Connect Remote Registry;CTRL+SHIFT+E="StartReg.exe" %TARGET%
In the above example I didn't pass the Path for StartReg.exe because I put it in my System32 directory which is, obviously, included in my % PATH% variable.
Note: Any download from our Free Utilities is not supported and is provided without warranty of any kind.
View an example of creating a Custom Tool within AA Console.
Photo by Robert Gaal
I've been working on on some code recently that deals with encryption and it always gives me the hiblijiblies. It's the kind of thing where you feel really secure right up to the point that you aren't secure any more. The biggest area of my worry is securing keys in software. In order for software to be able to encrypt and decrypt things on its own (i.e. without human intervention such as entering a password) the keys need to be in a place that the software can read them. And if the software can read them, then someone else is going to be able to. It can be obscured and hidden, but it can never be as secure as having a password known only in the mind of a user.
There are a couple of classic examples of this problem out there:
CSS or Content Scramble System is the encryption used on DVDs to prevent them being copied and played in unapproved geographical regions. It wasn't the strongest encryption in the world (it was 1995, afterall) but it wasn't until 1999 that someone broke it. Ostensibly to create a DVD player for Linux which didn't have an officially licensed player. Supposedly the initial project worked by disassembling a licensed software player to retrieve its keys.
AACS or Advanced Access Content System is the encryption used for HD-DVD and Blu-Ray in the same way as CSS. In an attempt to be better than CSS it adds a key revocation system. This allows content producers to create disks that won't work with known compromised keys thereby limiting the damage of an exposed key. It does seem to have worked somewhat better than CSS, but successful attacks against the system are out there.
Trusted Computing is an attempt to block the hole up for good. It's a set of hardware solutions which allow software to store and retrieve keys securely. Certainly better than software only, even it isn't immune to attacks.
For me there are three lessons from all this.
First, everything is a trade-off. You can add extra locks to your front door but that just makes it less convenient when you come and go. Finding the right balance is critical.
Second, diminishing returns are really steep when there is a weak link in the chain. Once you know where your weakest link is, it's much easier to determine when the diminishing returns curve makes any more effort pointless.
And last, it pays to always be paranoid. It's one thing to admit that you've realistically hit the limit of how secure you can make something, and it's another thing to give up entirely. There are always some small improvements that can be made and they should be explored as though the NSA has focused their resources like a laser on you personally, but only implemented if they realistically help out. Being paranoid doesn't mean never admitting that the "bad guys" sometimes win.
Photo by Apple
Since 2003 I've often made the statement: "I use a Mac; I make my money off of PCs."
When Adam and I customized the crap out of Tivoli Configuration Manager in 2003 - 2005 we built almost the entire solution on two macbooks (programming in Python). Keep in mind these customizations ended up managing over 16,000 PCs, 300+ Windows servers and about 100 Unix systems. Basically, OSX was in our enterprise, albeit quietly.
Apple's decision to cancel the XServe (announced last week and reported on CNET) has sparked a minor outcry from XServe users. According to Apple there is just not enough demand for the XServe. Hey, if there isn't enough demand then what are you going to do? This being said, it does make the job of any Sys Admin to successfully advocate for Apple products as "Enterprise Level" more difficult.
The obvious exception to the above statement is in relation to the iPhone or, perhaps better stated, Apple's mobile solutions. It's no mystery that Apple has been focusing on Mobile for quite a little while lately. If this is news to you you've probably spent too much time under your server room's raised floor. The big hurdle for Apple in enterprise is still to dethrone the Blackberry. Evertiq still predicts that Blackberry will be king of Mobile in business through 2015.
Apple didn't make much headway with XServe. I remember that Steve Jobs said in 2002, when the XServe was first announced, that Apple was getting into the enterprise server field "already humbled" (the exact phrase eludes me but Jobs got the point across that Apple didn't have sky-high expectations in this arena).
Not to worry, Sys Admins... Apple's presence will only increase in the enterprise, just not in the server room.
To get in the groove while I am scripting I have, historically, found it difficult to listen to audiobooks or podcasts. However, I realized this week that the ACE Broadcasting Network has been the only exception to this rule. I can (and do) listen to both Adam Carolla's and Larry Miller's podcasts while working and I don't get distracted. Hell Yeah.
I have used Microsoft SQL Server Express since 2005 and I am still a huge fan. If this was Facebook I would have, long ago, hit the Like button.
It seems like a lot of Sys Admins have two facebook accounts. A more sterilized one for family and co-workers and an uncensored one for the rest.
I think I am one of only 8 Sys Admins who has never owned, collected or even read, graphic novels.
A disproportionate number of my custom ringtones are taken directly from South Park though my single favorite ringtone is taken from The Big Lebowski (where Donny tells the Dude that his phone is ringing).
Screw what happens in elevators when no one is watching...think of what happens in Server Rooms when there is only one current occupant. ahem.
If your first response to the statement "We have a problem" is a variant of "We better have a meeting" then chances are I don't ever want to meet you.
It's been over a year since I first saw TheWebSiteIsDown and wanted to remind everyone how perfect is their first video, Sales Guy vs. Web Dude.
Follow me on Twitter @ShaneCorellian
Photo by Mr Thinktank
After many years of digging through the Windows Registry for various purposes, unbelievably not all of them purely masochistic, I have discovered a few keys that are amazingly well hidden. So well hidden, in fact, that many would claim that they don't exist. Ignore those people, they are just agents of the Triluminati Council on Bilderberg-Bohemian Relations and they want to keep these keys to themselves.
To see these keys you need a special version of Regedit.exe. I would point you to a download for it, but if I did it would be immediately removed by the powers that be. You may be able to get a copy from the Time Cube guy.
Hold on to your mad hats as we see how deep this rabbit hole goes.
This key enables the quantum components of your processor allowing you to decrypt everything like in that movie Sneakers. Normally only the NSA and Proctor and Gamble have access to this key.
This is where the most recent scan of your brain taken by your flat screen monitor is stored. This scan can be avoided by either wearing a tinfoil hat or by going back to a giant CRT monitor. The tinfoil hat would be less embarrassing, though.
Depending on where you bought your computer one or more items like these will be present. In typical government intelligence fashion, it appears that they don't follow the Windows development guidelines and store things that should never go in the registry. PGP backdoor decryption code should always go in .dll files.
Same as above, but only on really old computers.
Stored here are little bits and pieces of alien software which blue screened on your computer in the distant past. Shortly after initially displaying the contents of the key the system will rename it Crash\WeatherBalloon. Don't be fooled.
This key actually holds the real registry used by your computer, and it's very different than the "official" registry you've been told about. Apparently all blue screens, freezes, and odd behavior are coming from the system. Do you really think that a handful of hardware driver writers are capable of pulling it off? I don't think so.
If you see this key then you have been programmed as a sleeper assassin. Delete it immediately!
Set this key to DWORD 1 in order to prevent being Rick Rolled, as detailed in this tutorial.
Download your free copy of PDQ Deploy
Follow me on Twitter: @ShaneCorellian
OK, so PDQ Deploy Pro is released. What does this mean?
Our customers have requested the ability to schedule their software deployments. Gotcha. To schedule a deployment, simply create a PDQ Installer (the same as you would in the free PDQ Deploy) but instead of pushing the Deploy Now button, go to the Schedules tab and push the Add Schedule button. After entering your desired schedule, just save the Installer and your deployment is scheduled. It's that easy.
Define each deployment schedule
Schedule one or more deployments
Status of scheduled deployment
As you can see from the last screenshot, our deployment of Foxit Reader kicked off at 1:00 AM on Nov. 6. The duration of the installation took just over one minute. Of the 9 targets 8 were successful and one machine failed as it was offline at the time of the deployment.
Remember, with the Pro version of PDQ Deploy you can also throttle the bandwidth used for your deployments. This is accomplished via the PDQ Deploy Pro Configuration utility.
In the above example, our Pro performance is set to deploy to 30 targets concurrently. If you have more than 30 targets in a deployment, worry not, all this means is that PDQ Deploy Pro will select an initial 30 targets and as each of these 30 complete the next target in line will receive the deployment.
As you can have more than one deployment running at a time, you can also limit the total (across ALL deployments) concurrent targets.
You can see that each deployment will only utilize 50% of the available bandwidth on your network.
If you wish to receive (or have someone else receive) an email notification for each deployment, you simply specify the email addresses via the Notify tab of the Installer. (You need to have an email server specified in the PDQ Deploy Pro Configuration utility)
Don't think that we are just sitting back watching Dexter (actually, it's on right now... thanks TiVo) over here. New features for Pro are being developed right now. Future upgrades are included in PDQ Deploy Pro Maintenance Plans.
Here's a video demonstrating the scheduling feature of PDQ Deploy Pro.
As you were...
Follow me on Twitter @ShaneCorellian
A Database (DB) really runs at the heart of any decent Systems Management solution. SCCM/SMS stores its data, configurations and pretty much everything in a Microsoft SQL Server DB. Active Directory utilizes Microsoft's Extensible Storage Environment (ESE) which is tied to their Jet Database. IBM Tivoli can utilize different Relational Databases such as Oracle, DB2, SyBase and Microsoft SQL.
AA Console (formerly Admin Arsenal) uses a Microsoft SQL Compact Edition (CE) to store its data.
Well, PDQ Deploy Pro, which was released to beta last week, uses Microsoft SQL Server. To help our customers who may not have licenses for SQL Server we also work with Microsoft's free SQL Server Express.
We are also aware that many of our customers have to wear many hats in their respective Sys Admin duties. We want to make your SQL hat fit right away. This is why we are incorporating a Database Configuration tool into PDQ Deploy Pro.
We are less than two weeks away from releasing PDQ Deploy Pro. Below is a video demonstrating how we will automatically download, install AND configure your Microsoft SQL Server Express so that your PDQ Deploy Pro can be up and running in minutes.
Follow me on Twitter @ShaneCorellian
"A long dead soldier looks out from the frame
No one remembers his war; no one remembers his name
Go out to the meadow; scare off all the crows
It does nothing but rain here and nothing will grow"
All products will utilimately die. Period. But, like the death rattle of a man dying of terminal flatulence, some products hang on despite Kevorkian-like measures to invite death. I won't list the obvious candidates (Microsoft Vista, Microsoft Me, etc) but here is a special list of 3 massively hyped-up crap that, never should have made it past the third trimester.
Some say that Oracle's Network Computer (NC) was "before it's time". Ellison foresaw the Cloud. So what? If I offered In-flight movies before the Wright Brothers ever got to North Carolina I wouldn't be "ahead of my time" as much as I'd be psychic carny. Where was the infrastructure to support the NC? Where was the demand (in 1996) for network applications or SaaS? Let's face it, the Network Computer was only ever about Larry Ellison trying steal Gates' thunder. It had nothing to do with fulfilling a need or introducing some excellent content to the consumer. It was only about breaking the Windows "monopoly" which, in my opinion, Microsoft already excels at.
To paraphrase Homer: I've seen some [search engines] that sucked but these sucking sucks were the worst sucks that ever sucked". I had about as much chance of finding anything related to my search using Live.com as I had picking Adam's navel lint. The latter was also much more rewarding. Anyway, I'll hand it to Microsoft, BING doesn't suck nearly as bad as Windows Live Search but it still kinda has that Live stank attached to it. I didn't buy a Samsung Fascinate because it forced Bing as the search engine. (Yes, I know that this has been resolved but still, not even providing an avenue to Google?) I ended up, incidentally, buy a Samsung Galaxy Vibrant.
Tivoli Enterprise Console (T/EC)
OK, before any of my Tivoli friends get all aggro on me, I want to say that my beef is actually with the reliance that T/EC Rules had on Prolog.
I remember a T/EC instructor once told me that the reason IBM stuck with Prolog dependency in T/EC was that only Prolog allowed for the speed needed for processing all the Events that came into T/EC. I call B.S. on that. T/EC rule writers have benefited from the reliance on Prolog only because nobody knows or gives a crap about Prolog anymore. T/EC contracting positions are generally fairly lucrative because the 7 guys that know Prolog have been retired for 20 years.
If done right, T/EC actually kicked ass handling event management and correlation.
OK, I better get back to work. As you were...
follow me on twitter @ShaneCorellian
NOTE: JRE 6 update 23 has a different method of silent installation than update 22.
Regular Java updates can be a tedious chore for sys admins to deal with. Sometimes the updates can conflict with previous versions, or more likely, conflict with applications that were built for previous versions of the JRE.
Let's tackle the easy part; installing Java silently to multiple computers simultaneously. As we do with most of our video examples, we'll be demonstrating the installation with our free product, PDQ Deploy.
Download the updated JRE. We're using 6 update 22 in this example.
The file that you download is a self-contained .exe. Within the exe is the installation MSI. Oracle is continuing this approach. While it is possible to extract the MSI, it's not recommended. There are other actions that the EXE is taking.
Since it's not a native MSI, you won't be able to use the integrated MSI handling withing PDQ Deploy, however you can still pass MSI parameters for the deployment.
We have a video demonstrating the deployment of Java, as well as the method for determining the out-of-the-box parameters, or switches, that you can use to install Java silently and to suppress any required reboot.
Next step: troubleshooting. A user recently experienced the lovely and ever detailed error 1603 when deploying Java 6 update 22 using PDQ Deploy. As has been discussed on this blog, the 1603 error is generic, but it often means that the installation was expecting something that it didn't have access to, often a parameter or switch.
Be sure to use the correct switches. You can get the dialog for approved switches by running the JRE in a command window and entering a
before hitting Enter.
The switches to install JRE 6.22 are
If you're discovering that the installation is failing on some of your systems, be sure to turn on logging. You can do this using the following:
/quiet /norestart /l* c:\windows\temp\JRE-6.22.log
Java can be a beast to deal with sometimes. Our user discovered in their log file two different errors which had a lot of chatter on the Java forums. There were numerous work-arounds, but all of them had quite a bit of work involved. (Welcome to the world of Java.)
If you have questions regarding installing Java, or if you'd like to see video examples of other silent software installations, please let us know.
Follow me on Twitter @ShawnAnderson
Get your free copy of PDQ Deploy today.