There was a recent rash of car break-ins at the gym. Not an epidemic by any stretch, probably 4 or 5, but still...

My gym used to allow you to hang your keys from a peg board at the front desk. This way you could come to the gym dressed to work out, lock your valuables in your car, and not have anything to worry about. Ignorance is bliss.

The problem was that anyone who wanted to could go pick up your car keys, click the unlock button and find your car. Once there, they could rummage through your stuff and then walk back in and finish their workout as if nothing had happened. The people doing this were a little smatter then the average thief and would swipe some but not all of your cash leaving everything else in place. Most thieves would steal the whole car and be busted more quickly. The victims were unaware that anything had happened for several days.

Fortunately, once the victims realized what had happened, the gym was still able to pull security tapes and find out who was misbehaving. All of the bad guys were busted, and everyone can now breathe a sigh of relieve. It is once again safe to go to the gym.

Except there was still a fundamental problem. Putting your keys on a peg board by the front door is just asking for bad things to happen. One person got busted exploiting this security flaw. Others can still be exploiting it. In fact, others may well have been exploiting it and simply never got caught. How long would it take you to realize that $10 was missing from your wallet, if everything else was there? How would you even know when it went missing? Would you go to the front desk and even bother to ask them to review security tapes if you were only missing a small amount. Once highlighted, it is easy to see how commonly such vulnerability may have been exploited.

So the gym did the very reasonable precaution of removing the peg board.

To me the most shocking part of this story is the resulting uproar from gym members losing the convenient key peg. How dare they remove the trusted peg board? How can I work out now, I have to carry my keys from machine to machine? How can I enjoy my workout with this added inconvenience?

This all happened a couple of weeks ago, and some people are still complaining.

In light of the recent high profile hacking, there are a couple of parallels that can be drawn. Many web sites are riddled with vulnerabilities are crazy and easily exploitable as leaving your car keys by the front door while you work out. No one ever considered thanking the people who were swiping these keys for pointing out the vulnerability. Without a hesitation, they had their gym memberships revoked and are awaiting prosecution. The gym did recognize the vulnerability for what it is, and closed up that attack vector.

What can we learn from this?

Monitoring and logging will not prevent a crime but they will allow us to identify that a crime took place and may help track down who did it.

Once we find a security weakness, we need to eliminate it. We may never identify and eliminate all security weaknesses, but we cannot allow well known vulnerabilities to persist in our system.

In our case, we are not likely to meet resistance from end users. We are more likely to meet resistance from stake holders, product owners, keeper of schedules and budgets. We may meet resistance from integration partners, co workers, and third party vendors. Regardless of the source, we will see resistance, but the weakness needs to be dealt with.

There is no need to glorify a cracker for bringing to light a security weakness. Regardless of their claimed motives, they are not heroes. There is also no point in wasting time defending weaknesses once they are identified. Deal with the weakness and move on.

In may be embarrassing to find security weaknesses in our systems, but it is even more embarrassing to continue ignoring them. Even if it is unpopular, we need to seek out security weaknesses and eliminate them when we find them.

http://www.sans.org has put together the Common Weakness Enumeration http://cwe.mitre.org/ which lists out common weaknesses. The site navigation takes a little getting used to, but there is a treasure trove here. Here is the detail page for SQL Injection. It clearly states how this can be exploited, in case anyone doubts that the weakness should be taken seriously, and more importantly how to mitigate the risk.

I have recently been exploring the wonders of PowerShell. I also recently worked through some deployment / configuration issues and thought I would bring my newly acquired appreciation of PowerShell to the mix.

I had never given PowerShell a second thought. I really had no idea what it was or how it was used. This is very surprising given my perl background. Once I gave it a chance, I loved it.

One of the most exciting things about PowerShell is that unlike with regular scripting languages, you are not dealing with text. You are dealing with objects. More specifically, you are dealing with DotNet objects. So when you run a cmdlet like get-service, you don't get text back, you back a ServiceController. This means that you can easily start and stop services:

$service = get-service $serviceName

if ($service –ne $null)

{

$service.Stop()

}

 

This is very nice and so much easier being able to deal with objects.

With this in mind, how do we deal with the config files?

Instead of simply dealing with text, we can deal with an xml document. Powershell will actually "strongly type" the document, sort of. Consider a script like this:

$xml = [xml](get-content($path))

$xml.Save($path+ ".old")

$runtime = $xml.configuration["runtime"]

if($runtime.gcServer -eq $null)

{

$gc = $xml.CreateElement("gcServer")

$gc.SetAttribute("enabled", "false")

$runtime.AppendChild($gc)

}

$runtime.gcServer.enabled = "false";

$xml.Save($path)

 

This lovely little bit of magic will add the following configuration setting if it does not already exist

 

<configuration>

<runtime>

<gcServer enabled = "false"/>

</runtime>

</configuration

 

There is a subtle point to note here. If there are no child elements to runtime to start with, $xml.configuration.runtime will be a string. If runtime already has elements, it will be an xml element. $xml.configuration["runtime"] will always be an xmlelements. This subtle distinction caused me some confusion initially and complicated writing my little script.

 

This type of script makes it easy to make a change to a deployment target, especially if you must make the same change to several servers.

It also makes it easy audit and verify that the server is in fact configured properly.

 

Turns out there are lots of uses for this new (at least new to me) tool for the ever growing tool box.

 

What are you doing with power shell?

I will be speaking at the Carolina Code Camp this weekend http://codecamp.developersguild.org/.

There will be 45 sessions by 30 different speakers spread over 9 tracks. It should be a good time.

The biggest problem for anyone attending is deciding which sessions to go to. With 9 tracks there will be many great sessions going on at the same time.

From a presenter perspective, that ups the ante. I am very much aware that when I am speaking there are 8 other folks that you could be hearing instead.

My presentation is titled "Getting Back in the Game". The target audience is Recent Graduates, Developers who have been out of the industry for a while, IT professionals looking to make the transition to a developer, as well as active developers who want a different type of job (web developer looking to get into Silverlight for example).

In this session, we will explore the problem of "How do you get opportunities without experience" and "How to get experience without opportunities".

Much of the thought behind this material comes from personal reflections while I sat in on recent interviews. It was very frustrating watching candidates flounder over the questions of having relevant experience and thinking about what they could do to shore up this weakness.

If you fit into this target audience, I hope you will come see the presentation and hopefully get some fresh insight.

If make hiring decisions, maybe you can join the discussion and add your perspective to the conversation.

Whether you come to my session or not, there will be a lot of great material discussed all day long.

Much has been said recently about Red Gate's community involvement and commitment to the DotNet community. Much of this has been unduly negative.

Before you start throwing stones and spewing obscenities, consider some additional facts:

  • Red Gate's software is actually very good. I have worked on many projects where Red Gate's software was instrumental in finishing successfully.
  • Red Gate is VERY good to the community. I have spoken at many user groups and code camps where Red Gate has been a sponsor. Red Gate consistently offers up money to pay for the venue or food, and they will often give away licenses as door prizes. There are many such community events that would not take place without Red Gate's support. All I have ever seen them ask for is to have their products mentioned or be listed as a sponsor. They don't insist on anyone following a specific script. They don't monitor how their products are showcased. They let their products speak for themselves.
  • Red Gate sponsors the Simple Talk web site. I publish there regularly. Red Gate has never exerted editorial pressure on me. No one has ever told me we can't publish this unless you mention Red Gate products. No one has ever said, you need to say nice things about Red Gate products in order to be published. They have told me, "you need to make this less academic, so you don't alienate too many readers. "You need to actually write an introduction so people will know what you are talking about". "You need to write this so that someone who isn't a reflection nut will follow what you are trying to say." In short, they have been good editors worried about the quality of the content and what the readers are likely to be interested in.

For me personally, Red Gate and Simple Talk have both been excellent to work with.

As for the developer outrage…

I am a little embarrassed by so much of the response that I am seeing. So much of the complaints remind me of little children whining "but you promised"

Semantics aside. A promise is just a promise. It's not like they "pinky sweared". Sadly no amount name calling or "double dog daring" will change the economics of the situation.

Red Gate is not a multibillion dollar corporation. They are a mid size company doing the best they can. Without a doubt, their pockets are not as deep as Microsoft's. I honestly believe that they did try to make the "freemium" model work. Sadly it did not.

I have no doubt that they intended for it to work and that they tried to make it work. I also have no doubt that they labored over making this decision. This could not have been an easy decision to make.

Many people are gleefully proclaiming a massive backlash against Red Gate swearing off their wonderful products and promising to bash them at every opportunity from now on. This is childish behavior that does not represent professionals. This type of behavior is more in line with bullies in the school yard than professionals in a professional community.

Now for my own prediction…

This back lash against Red Gate is not likely to last very long. We will all realize that we still need their products. We may look around for alternatives, but realize that they really do have the best in class for every product that they produce, and that they really are not exorbitantly priced. We will see them sponsoring Code Camps and User Groups and be reminded, "hey this isn't such a bad company".

On the other hand, software shops like Red Gate, will remember this back lash and give a second thought to supporting open source projects. They will worry about getting involved when an individual wants to turn over control for a product that they developed but can no longer support alone. Who wants to run the risk of not being able to follow through on their best intentions.

In the end we may all suffer, even the toddlers among us throwing the temper tantrum, "BUT YOU PROMISED!"

Disclaimer

Before anyone asks or jumps to conclusions, I do not get paid by Red Gate to say any of this. I have often written about their products, and I have long thought that they are a wonderful company with amazing products. If they ever open an office in the SE United States, I will be one of the first to apply.

I have heard a lot of outrage over Red Gate's decision to charge for Reflector.

In the interest of full disclosure, I am a fan of Red Gate. I have worked with them on several usability tests. They also sponsor Simple Talk where I publish articles. They are a good company.

I am also a BIG fan of Reflector. I have used it since Lutz originally released it. I have written my own add-ins. I have written code to host reflector and use its object model in my own code.

Reflector is a beautiful tool. The care that Lutz took to incorporate extensibility is amazing. I have never had difficulty convincing my fellow developers that it is a wonderful tool. Almost always, once anyone sees it in action, it becomes their favorite tool.

This wide spread adoption and usability has made it an icon and pivotal pillar in the DotNet community. Even folks with the attitude that if it did not come out of Redmond then it must not be any good, still love it.

It is ironic to hear everyone clamoring for it to be released as open source. Reflector was never open source, it was free, but you never were able to peruse the source code and contribute your own changes. You could not even use Reflector to view the source code. From the very beginning, it was never anyone's intention for just anyone to examine the source code and make their own contributions aside from the add-in model. Lutz chose to hand over the reins to Red Gate because he believed that they would be able to build on his original vision and keep the product viable and effective. He did not choose to make it open source, hoping that the community would be up to the challenge. The simplicity and elegance may well have been lost with the "design by committee" nature of open source.

Despite being a wonderful and beloved tool, Reflector cannot be an easy tool to maintain. Maybe because it is so wonderful and beloved, it is even more difficult to maintain.

At any rate, we have high expectations. Reflector must continue to be able to reasonably disassemble every language construct that the framework and core languages dream up. We want it to be fast, and we also want it to continue to be simple to use. No small order.

Red Gate tried to keep the core product free. Sadly there was not enough interest in the Pro version to subsidize the rest of the expenses. $35 is a reasonable cost, more than reasonable.

I have read the blog posts and forum posts complaining about the time associated with getting the expense approved. I have heard people complain about the cost being unreasonable if you are a developer from certain countries.

Let's do the math. How much of a productivity boost is Reflector? How many hours do you think it saves you in a typical project? The next question is a little easier if you are a contractor or a consultant, but what is your hourly rate? If you are not a contractor, you can probably figure out an hourly rate. How long does it take to get a return on your investment?

The value added proposition is not a difficult one to make.

I have read people clamoring that Red Gate sucks and is evil. They complain about broken promises and conflicts of interest. Relax! Red Gate is not evil. The world is not coming to an end. The sun will come up tomorrow.

I am sure that Red Gate will come up with options for volume licensing or site licensing for companies that want to get a licensed copy for their entire team. Don't panic, and I am sure that many great improvements are on the horizon. Switching the UI to WPF and including a tabbed interface opens up lots of possibilities.

There has been much news about the hack of the Gawker web sites. There has even been an analysis of the common passwords found.

This list is embarrassing in many ways. The most common password was "123456". The second most common password was "password".

Much has also been written providing advice on how to create good passwords. This article provides some interesting advice, none of which should be taken.

Anyone reading my blog, probably already knows the importance of strong passwords, so I am not going to reiterate the reasons here.

My target audience is more the folks defining password complexity requirements. A user cannot come up with a strong password, if we have complexity requirements that don't make sense.

With that in mind, here are a few guidelines:

 

Long Passwords

Insist on long passwords. In some cases, you may need to change to allow a long password. I have seen many places that cap passwords at 8 characters. Passwords need to be at least 8 characters minimal. Consider how much stronger the passwords would be if you double the length. Passwords that are 15-20 characters will be that much harder to crack. There is no need to have limit passwords to 8 characters.

Don't Require Special Characters

Many complexity rules will require that your password include a capital letter, a lower case letter, a number, and one of the "special" characters, the shits above the number keys. The problem with such rules is that the resulting passwords are harder to remember. It also means that you will have a smaller set of characters in the resulting passwords. If you must include one of the 9 digits and one of the 9 "special" characters, then you have dramatically reduced the character set that will make up the final password. Two characters will be one of 10 possible values instead of one of 70. Two additional characters will be one of 26 possible characters instead of a 70 character potential character set. If you limit passwords to 8 characters, you are left with only 7 characters having the full set of 70 potential values.

With these character restrictions in place, there are 1.6 x1012 possible passwords. Without these special character restrictions, but allowing numbers and special characters, you get a total of 5.76x1014 possible passwords. Even if you only allowed upper and lower case characters, you will still have 2.18X1014 passwords.

You can do the math any number of ways, requiring special characters will always weaken passwords.

Now imagine the number of passwords when you require more than 8 characters.

 

If you are responsible for defining complexity rules, I urge you to take these guidelines into account. What other guidelines do you follow?

Let's continue exploring some of the SQL Smells from Phil's list. He has been putting together.

Datatype mis-matches in predicates that rely on implicit conversion.(Plamen Ratchev)

This is a great example poking holes in the whole theory of "If it works it's not broken" Queries will this probably will generally work and give the correct response. In fact, without careful analysis, you probably may be completely oblivious that there is even a problem. This subtle little problem will needlessly complicate queries and slow them down regardless of the indexes applied.

Consider this example:

CREATE TABLE [dbo].[Page](

    [PageId] [int] IDENTITY(1,1) NOT NULL,

    [Title] [varchar](75) NOT NULL,

    [Sequence] [int] NOT NULL,

    [ThemeId] [int] NOT NULL,

    [CustomCss] [text] NOT NULL,

    [CustomScript] [text] NOT NULL,

    [PageGroupId] [int] NOT NULL;

 

CREATE PROCEDURE PageSelectBySequence

(

@sequenceMin smallint ,

@sequenceMax smallint

)

AS

BEGIN

SELECT [PageId] ,

[Title] ,

[Sequence] ,

[ThemeId] ,

[CustomCss] ,

[CustomScript] ,

[PageGroupId]

FROM [CMS].[dbo].[Page]

WHERE Sequence BETWEEN @sequenceMin AND @SequenceMax

END

 

Note that the Sequence column is defined as int while the sequence parameter is defined as a small int. The problem is that the database may have to do a lot of type conversions to evaluate the query. In some cases, this may even negate the indexes that you have in place.

Using Correlated subqueries instead of a join   (Dave_Levy/ Plamen Ratchev)

There are two main problems here. The first is a little subjective, since this is a non-standard way of expressing the query, it is harder to understand.

The other problem is much more objective and potentially problematic. You are taking much of the control away from the optimizer. Written properly, such a query may well out perform a corresponding query written with traditional joins. More likely than not, performance will degrade.

Whenever you assume that you know better than the optimizer, you will most likely be wrong. This is the fundmental problem with any hint.

Consider a query like this:

 

SELECT Page.Title ,

Page.Sequence ,

Page.ThemeId ,

Page.CustomCss ,

Page.CustomScript ,

PageEffectParams.Name ,

PageEffectParams.Value ,

( SELECT EffectName

FROM dbo.Effect

WHERE EffectId = dbo.PageEffects.EffectId

) AS EffectName

FROM Page

INNER JOIN PageEffect ON Page.PageId = PageEffects.PageId

INNER JOIN PageEffectParam ON PageEffects.PageEffectId = PageEffectParams.PageEffectId

 

This can and should be written as:

 

SELECT Page.Title ,

Page.Sequence ,

Page.ThemeId ,

Page.CustomCss ,

Page.CustomScript ,

PageEffectParams.Name ,

PageEffectParams.Value ,

EffectName

FROM Page

INNER JOIN PageEffect ON Page.PageId = PageEffects.PageId

INNER JOIN PageEffectParam ON PageEffects.PageEffectId = PageEffectParams.PageEffectId

INNER JOIN dbo.Effect ON dbo.Effects.EffectId = dbo.PageEffects.EffectId

 

The correlated query may just as easily show up in the where clause. It's not a good idea in the select clause or the where clause.

Few or No comments.

This one is a bit more complicated and controversial. All comments are not created equal. Some comments are helpful and need to be included. Other comments are not necessary and may indicate a problem.

I tend to follow the rule of thumb that comments that explain why are good. Comments that explain how are bad. Many people may be shocked to hear the idea of a bad comment, but hear me out.

If a comment is needed to explain what is going on or how it works, the logic is too complex and needs to be simplified.

Comments that explain why are good. Comments may explain why the sql is needed are good. Comments that explain where the sql is used are good. Comments that explain how tables are related should not be needed if the sql is well written. If they are needed, you need to consider reworking the sql or simplify your data model.

Use of functions in a WHERE clause. (Anil Das)

Calling a function in the where clause will often negate the indexing strategy. The function will be called for every record considered. This will often a force a full table scan on the tables affected.

Calling a function will not guarantee that there is a full table scan, but there is a good chance that it will. If you find that you often need to write queries using a particular function, you may need to add a column to the table that has the function already applied.

SQL Tips for Learning a new Database

 

The hardest part of writing SQL statements is in understanding the database that you are working with. The raw mechanics of SQL for the most part are fairly straight forward. Often times really complex SQL statements point to business logic showing up where it doesn’t belong.

The hard part about learning a database is the lack of consistency and documentation. Data diagrams are often of little help. Diagrams are good for an overview and to get a list of tables, etc, but when the database has hundreds of tables with some tables having dozens of columns; the diagram can be a little overwhelming. Sometimes, a data diagram may not be helpful because all of the relationships are not actually defined. Poor design or data integrity problems often mean that foreign keys are implied but not enforced by the database. A data diagram will offer little help in finding such relationships, but you may still be able to ferret out such relationships through the system tables.

Every database includes a data dictionary that the database server uses to keep everything straight. We can use this data dictionary to our own benefit.

Instead of simply viewing an alphabetical list of tables, we can view all of the tables that include a given column or a given combination of columns or include these two columns but not a third column. We can easily get a list of all of the tables that include any columns that are of a particular data type or have more than a given number of columns or less than a given number of columns. There is a lot of potential here.

For SQL Server, the main tables that you will be interested in are in the information_shcmea views. These views exist in every database, but they are only visible in the master database. To make most of this magic work, we will really be interested in just a couple of views. Tables and Columns.

Information_Schema.Tables will give us some details about all of the tables in a given database.

Information_Schema.Columns will give us some details about of the columns in a given table.

With these two views, we can write queries such as:

Select * from information_Schema.tables where table_name like ‘%blah%’

Select * from information_Schema.columns where column_name = ‘Member_ID’

Get a list of tables that include a Member_ID column, a FirstName column and any DateTime column and at least 20 columns total.

Select * from information_schema.tables where table_name in (

Select table_name from information_schema.columns where column_name = ‘Member_ID’)

And table_name in (

Select table_name from information_schema.columns where column_name = ‘FirstName’)

And table_name in (

Select table_name from information_schema.columns where datatype = ‘DateTime’)

And table_name in (

select table_name from (select table_name, count (1)

From information_schema.columns

Group by table_name

Having count(1) > 20)

)

As you can see the possibilities are limited only by your imagination.

Oracle provides the same functionality. Actually every relational database must. For Oracle, the view in question would be all_tables and all_tab_columns.

In honor of Phil Factor's recent post on SQL Smells, I thought I would post some blogs SQL smells and why some of them are bad as well as when they may be OK.

Let's start with the top of the list that Phil has compiled.

Use of deprecated syntax such as *= (Dave Howard)

There are two problems here. First this is not in compliance with the SQL 92 standard. We should strive our code to be standards compliant whenever there is a standard available. Now this sounds like a cop out along the lines of "Because I said so" but there is a better reason.

This join syntax was standardized as it was to reduce ambiguity. One source of ambiguity stems from not being able to express a full outer join with the old syntax. It is confusing to have the join conditions blended in with the filtering conditions in the where clause. The SQL 92 syntax provides strong separation.

Denormalisation that requires the shredding of the contents of columns. (Merrill Aldrich)

This description could apply to any of several potential problems. I am not sure which problem Merrill was specifically referring to or what examples he finds most egregious.

I will speak here about the perils of so called "smart columns" These are columns that need to be parsed to get their true meaning. There are many examples. I worked on a system once that was riddled with so called CSZ fields. These were fields that stored the City State and ZipCode in a single field. You had to parse these fields to get the details because they were not stored in separate fields. Clearly a very bad design.

I also feel the same way about storing XML data in a varchar(max) field. If you must store XML data in the database, use the XML data type. XML data type will provide support for verifying that the content is XML and in the format expected.

Contrived Interfaces

I am not sure exactly what whoever submitted this was getting at, but I am going to take this opportunity to talk about naming conventions. There are other smells that talk about prefixing, so I will talk about how you name the database objects apart from the skipping the prefixes.

Bad naming conventions lead to confusion, making the database more difficult to work with and harder to understand. There is no need to use contrived abbreviations.

FirstName is dramatically easier to read and follow then F_NME.

Especially in SQL server the maximum size for object names is larger than you will ever need and the database preserves case, there is no need sprinkle names with underscores to separate words or shorten the names. Separate words with case changes and spell the names out. It will cut down on confusion.

Use of deprecated datatypes such as TEXT/NTEXT (Dave Howard)

You probably cannot go through your database and change every data type that needs to be changed. When you get advanced warning that a data type is going away, stop using it.

As you can, change your data types. It is better to go through and update individual fields as you can than to have to scramble when you try to upgrade the database version. Upgrading the database version can be daunting enough without such extra complications.

 

This covers my thoughts on the first four smells in Phil's list. Future posts will flush out my thoughts on others.

 

I would love to hear your thoughts on these smells.

 

    

Phil Factor blogged about his quest for SQL Code Smells. He wants to start some discussions about coding standards and best / worst practices.

This is sure to generate a lot of discussion. In fact it already has.

Follow the #sqlcodesmells hashtag and you can follow the discussion. I have tweeted several sql code smells of my own.

There are also some interesting comments on the blog. There is a debate brewing there about the futility of defining standards, art vs. science arguments, etc. One comment even goes so far as "On many a project I have seen coders conform to the arbitrary rule set only to write poor, hard to maintain code."

This often happens, but it does not diminish the importance of such rules. You also can't legislate morality, but we still try.

But such comments avoid the whole point of Code Smells. Code smells are not meant to be dogmatic. They are guidelines meant to point out that there might be something worth investigating.

Code Smells should not be viewed as "arbitrary rules".

Look back over the original Code Smells from Kent Beck and Martin Fowler. They all focus on taking out the antagonism and interjecting a sense of humor.

I have always found that this makes code reviews just a little easier.

I urge everyone to check out Phil's site, contribute your own smells. Approach the whole process with a sense of humor and have fun with it.

Add your comments to Phil's blog or tweet with the #sqlcodesmells hashtag. I would love to hear everyone's thoughts.

What are your most annoying SQL practices? Biggest Pet peeves? Worst practices?

Recently I spoke to my daughter's first grade class about what I do for a living. As a software architect, it is hard to explain what I do for a living. Even among "technical" people, this is difficult to explain.

How do you explain this to First Graders?

I scrambled thinking on my feet trying to come up with an example that I felt First Graders could relate to.

A software system is like the playground at recess. The playground is filled with various programs competing for the slides and swings and their turn in the sandbox.

So far so good!

When everyone plays fair, everyone gets a turn and can enjoy all of the playground equipment and have a good time. One of the many roles for a software architect is to ensure that everyone plays fair and that no one is being a bully.

A bully may damage the playground equipment or refuse to let others take their turn, or keep others from being able to enjoy their turn on the equipment. Someone has to protect everyone from the bullies.

We build in fault tolerance to ensure that a bully doesn't damage the equipment. We design distributed systems to ensure that the playground can scale to accommodate everyone. We also rely on preemptive operating systems and dead lock detections to ensure that individual processes are taking turns.

Turned out it was a pretty good way to talk about what I do.

One of the most intriguing topics in computer security is defense in depth. This basically means having many layers to security. Think about the walls and defenses in a medieval castle. An intruder has to storm each of the outer walls before they could get access to the castle. This is the same logic that leads to system designs with firewalls, intrusion detection systems, demilitarized zones, etc.

The idea is that if one layer is breached then other layer may catch and stop the intrusion. More likely, having to breach each layer may slow the intrusion down to allow professionals time to catch the intruder in the act.

As a developer, you may think this looks like a bunch of stuff for the infrastructure guys. Looks can be deceiving.

There is a great deal to be done from a development perspective to augment the work already being done. The most important area for us to focus on is data validation, logging, and exception management. Most developers finally understand the importance of validating user input, but how many of us validate anywhere else? Do you validate data read from the database? Data read from the registry? Data read from a config file?

This is defense in depth. Consider UI input validation the otter most wall. In many ways this is a convenience for the users who are using your application as intended. Validations in the business layer are another wall. Validations in the data access layer are another wall. Validations in the database schema are a final wall. Each wall is important, and each wall should not negate the need to do validations on the next.

Now for an added twist, what do you with your validation results? Specifically, what do you do when an input validation fails? Do you log it anywhere? Are you tracking it?

Most likely, you reject the input. You probably complain to the user explaining what the correct input should be, but do you log these details anywhere? Logging failed input validations becomes a critical part of intrusion detection. Seeing that someone attempted a SQL injection attack gives you advance warning that someone is probing your system for vulnerabilities. Logging such details and keeping track of these logs may reassure you that your validations are working. It may also give you an indication of where future attacks may originate from. Every piece of information can be helpful.

We have also heard repeatedly about the perils of displaying system information in exception details. We all know not to display too much information to the user and risk giving away details like file locations, database location, database version, code details, etc. Sometimes even the very fact that an exception was thrown can give an intruder all the details they need. Have you heard of blind sql injection? We can also this information to our advantage. Whenever an exception is thrown, chances are that someone is using our application in a way that we did not predict. By reviewing the exception logs, we may be able to see patterns of unexpected use. This may alert us to someone attempting something like a blind sql injection or trying to overflow a buffer or trying to break the system to see what details they can find. With a good exception management strategy in place, they are also letting us know where the next attack may come from and what form it may take.

Seen from this perspective, mundane tasks like input validation and exception management suddenly take on a much more exciting, exotic feel.

Cyber security, Cyber war, Cyber vulnerabilities are all hot topics in the news right now. They should be. Most applications and our very infrastructure are incredibly vulnerable. This should remind of us of Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. A swarm of wood peckers may be closer than we think.

Everyone gets outraged about privacy vulnerabilities with google and FaceBook without ever thinking about how vulnerable their privacy is nearly every time they connect to any web site.

It makes national news (often international news) when personal information is lost in a high profile case, but this is often too little too late.

Many people read headlines about credit cars being stolen and feel powerless and helpless. The feeling is easily understood. As programmers, we are also often just as powerless.

From a security perspective, even if we do everything right, our applications are still vulnerable, but this should never be an excuse to not take ever step that we can to secure our applications or push for security on every application that we work with. This should inspire us to be more vigilant and protective.

There are a couple of points worth pointing out and remembering:

  • If an application is usable, it is vulnerable.
  • The myth of the secure application needs to be forgotten. Our goal needs to be defensible applications.
  • Computer intrusions are inevitable. Security counter measures will not prevent an intrusion. They will hopefully help us survive the intrusion. Think about how a seat belt does not prevent an accident but it helps you survive an accident.
  • If you are a computer security professional, developers are not the enemy.

The only secure computer is the one locked in a room with no key and no network connection. It won't be very useful, but it won't be compromised. The sad reality is that if the right individual or group wants to compromise your system, they will. Most systems seem safe simply because they have not attracted the wrong attention. An easy way to attract this wrong attention is to brag about how secure you are. Many hackers are drawn to the challenge of the "secure" system.

Don't ever think about your system as secure. Believing that you system is secure will draw unwanted attention to your system as well as lull you into a false sense of security. Accept the fact that it is vulnerable and requires constant monitoring and protection. Every safe guard that you put in place will slow an intrusion down, may allow you to identify and catch whoever is attempting the intrusion, but it will not prevent the intrusion.

We need to think about the automobile industry and their safety features. There was a time when the conventional wisdom was that the best way to survive a car accident was to not have an accident. To that end, the focus was on blaming the victim. "If you drove better, you wouldn't be injured." We often have a similar reaction with computer security today. "If you had a better password …" "If you had better firewall rules …" "If you had dedicated servers …" "If you were up-to-date on your patches …" "If you conducted penetration tests …" "If you had validated your input …" "If you had …"

These statements are all true, but even with a strong password behind a properly configured firewall on dedicated servers that are fully patched; the right hacker can still break through. Even if I have anti lock brakes, electronic stability control, and two dozen airbags while wearing a seat belt, I can still have an accident. The difference is that the same accident in a Model T will result in my death while in a modern "safe" car will result in a pricy repair bill for the car.

Strong passwords, firewalls, etc won't keep someone from compromising your system, but they might slow them down enough to allow you to catch them in the act. In essence, they won't prevent the intrusion, but they may help you survive.

Finally, I want to touch base on the role that developers can serve in computer security. We are not the enemy. I heard a security professional make the assertion that every system vulnerability exists because a developer messed up. This is the wrong attitude to take. Very rarely, will a developer set out to intentionally compromise the system that they are working. Most developers are filled with too much pride to even consider that. The developers designing and building a system should be viewed as the single most important strategic partner in securing the application.

Among the many roles that the application and specifically the UI serve is to be an application level firewall controlling access to the database. Viewed from this perspective, their importance becomes obvious. When a developer writes input validations, they are writing firewall rules that could never be defined anywhere else. Exception management logic and logging becomes important intrusion detection mechanisms that could not be duplicated in any way. Every business rule implemented serves to further strengthen the integrity of the system.

I hope to flush out more thoughts on computer security and how as developers we can strengthen our systems and better defend the worlds we create with our code.

I recently re styled my blog. I wanted a new look and something unique. I played with the various themes that you can chose from when customizing your blog. I also saw that you can add custom CSS, so I knew that it was possible to get some unique looks.

AI started by adopting the naked skin which applies no style sheets. This gave me a blank slate to work with. It also looks very difficult to read:

So I now have a blank slate to work with.

I examined the source for the page to look at the structure. I discovered a few key ids. Container, navigation, header, content, and so on. I knew I had what I needed to make the blog look the way I wanted.

This would only require a little experimenting. Naturally, I would prefer to experiment in private. For me at least, style sheet voodoo is as much art as it is science. Experimenting will be a lot of trial and error. I would rather everyone not see all of the intermediate results.

So I view source and save the html locally. In order to make everything work right, I have to expand out the local references. An href like this: href="/skins/_System/commonstyle.css"> Becomes this: href="http://geekswithblogs.net/skins/_System/commonstyle.css">

 

So far so good.

Next I created an empty style sheet and linked in into my saved html file. The CSS that I place in this file will eventually go into the custom CSS in my blog configuration.

 

Now let the fun begin.

 

The first thing that I decided to do was to style the actual container:

#container

{

margin-top:40px;

    width :800px;

    background-color:#DCDCDC;

    margin-left :auto ;

    margin-right :auto;

}

 

Next, I styled the navigation area:

#navigation

{

width :200px;

padding-left :10px;

padding-right :10px;

border : solid 1px black;

border-right :solid 1px white;

display:block;

background-color:#6B7289;

}

 

And then the content area:

#content

{

width : 550px;

padding-left :10px;

padding-right :20px;

background-color:#8DA4C3;

}

 

At this point, the blog looks much more readable, but there are still many details. We begin to see the color scheme take shape as well as the structure of the document. Now we can move on to some more nitpicky details involved in getting just rights. Now let's turn our attention to the titles.

 

#navigation .title

{

font-weight :bold;

background-color :black;

color :White;

padding-left :-10px;

text-align:center;

 

}

 

#header .title

{

font-size :3em;

text-align:center;

font-family :Impact

padding-top:15px;

}

#header .title a

{

    color:#59718F;

}

#header .subtitle

{

    font-size :.8em;

    text-align :right;

    margin-right : 10px;

    margin-bottom :10px;

}

 

#content .title

{

    font-size :1.2em;

    line-height : 3em;

    text-align :center ;

    background-color :#59718F;

    margin : 3px;

    background-color:#ff7c11;

}

 

#content .title a

{

    color :White;

}

 

 

 

Now we are looking a lot like the final site, but there are still some details missing. Details that may be easily overlooked, but they are important details none the less. There is too much space between the items in the lists, and we would love to lose the bullet points. I also want to include my image to be included in the title. Part of my branding.

#container

{

background-image :url(http://tinyurl.com/2v6f5la);

background-position:top left;

background-repeat:no-repeat;

border : solid 1px black;

min-height:100px;

}

 

The fun is just beginning. Now to play with adding some jQuery magic

I recently had to introduce a DTO pattern on a project. The application was split between a server side and a desktop side using http based WCF to handle the communication.

The project had been working well passing NHibernate entities back and forth, but there was too much data being transferred.

So we introduced the DTO pattern. I added a GetDto method to the base class for the Entities. I added a GetEntity to the base class for the DTOs. We created DTO objects for the entities that we wanted to switch over.

We spent a little bit of time debugging and working out the nuances in the GetDto and GetEntity methods. Everything was working fine. We modified the service classes and proxies to expect the DTO instead of the Entity. Everything is looking fine.

Now we try to pass data back and forth …

"The connection was closed unexpectedly"

 

We start debugging. We can step into the code on the desktop side. Everything looks fine. We step into the code on the server side. Everything looks fine. We finish the method on the server side, and we are slapped in the face with "The connection was closed unexpectedly".

I search all over the place for possible causes. I blame WCF configuration. I blame the alignment of the stars. I blame the weather. I blame the driver who cut me off in traffic.

Stubbornly, I refuse to blame the DTOs. This is a simple pattern. I have used it before to solve this very problem. Surely they cannot be the problem.

I switch back to passing Entities. The closing connection problem goes away. This lets everyone off the hook. Except for that driver. I will find a way to blame him yet.

So it has to be a problem with the DTO. I am stumped…

I go back to looking at the Entity and comparing it to the DTO. Both have the DataContract attribute. Both have DataMember attributes for every property. They do not appear to be any meaningful differences I explicitly call serialize on the DTO. It serializes without a problem.

I am still stumped. On a lark, we decide to try changing the base class of the DTO to match the base class of the Entity.

This is when we discover that EntityBase has a DataContract attribute, and DTOBase does not. We make the change.

The clouds part. The angels sing! Amazingly problem solved!

This should not have caused the problem. The error message should have been much better and definitely point us in a good direction, but what a relief that we figured it out.

Hopefully this will save someone else some trouble shooting time.

Now what can I blame on that driver …