Why are “mandatory” fields baaad?

Last time, I gave you a recipe on how to do a bad thing. If you really need to do something bad, I figure it’s better to do it the way that would hurt the least, or the way that isn’t so bad after all. I wouldn’t have dreamt of a comment on that blog post, but Ian came, and gave the perfect example which really explains why things like that *ARE* bad. Funny thing is, how many customers really go for such a requirement, and even funnier, how many consultancies give in. I gave in once, most of you probably did. But, why are mandatory fields so bad after all?

First of all, what are mandatory fields? Go, ask yourself. How do you really define them? If I were a customer, I’d probably go somewhat like this: Mandatory fields are those fields that must contain a value before you can perform operations that depend on those values. With a rationale like this, it is pretty obvious that you better make sure users enter values into these fields before trying to do something, otherwise bad things may happen. But NAV (and many other ERP systems as well) don’t just prevent you from entering partial information before going on, because preventing users from going on is the worst productivity killer, and faulty data generator, out there.

Say you need to enter twenty customers into the system. You open the customer card, enter the information for the first customer, then you want to go on defining another one, but the system gives you an ugly error message letting you know that sorry you can’t continue because you didn’t enter some important piece of information. (No, this won’t really happen to you in NAV, unless someone messed with it badly, this is just for illustration.) So, something like this happens, and you can’t continue because, say, you didn’t enter the Gen. Bus. Posting Group. Oh, my! What in the earth is the gen. bus. posting group? You don’t know, because you are not an accountant, you don’t know a thing about posting, you only know that by 10 AM you need to have these twenty customers in the system, and now this stupid system is stopping you. So what you do? You choose some gen. bus. posting group just to be able to go on. But then the system asks you about a Location Code. So you just choose any, because otherwise you wouldn’t be able to do your job. You just do it for any of the fields the system bugs you about. 10 AM, mission completed, over and done.

Then salespeople start posting, end start getting all sorts of errors, such as items not being available in location such and such, or even worse – no error happens, but the systems posts transactions happily to incorrect accounts, because an incorrect General Business Posting Group was specified in the customer card by someone who shouldn’t have come close to this field in the first place.

A proper ERP system, such as NAV is, doesn’t work like this, and it is better not to try to make it work like this, because from technical perspective, it can be a nightmare to implement it, and it introduces more problems than it solves. So, let’s go back to our mandatory fields question. How do you deem a field mandatory? A company doing full-blown logistics, must have several locations and track their items through all these locations, might even have customers coming to various default locations, so they probably can’t live without entering the Location Code information in their customer cards. So this must be a mandatory field for them, right? Well, kinda so. But for a company doing only sales, who don’t really have locations, and even if they do have a warehouse of a sort, they have only one, and they don’t need to track anything by locations. So for these guys, Location Code shouldn’t be a mandatory field. If you have logistics, knowing the unit of measure is, of course, extremely important, but a company caring only about financial aspect of their inventory can really live without knowing it. So, is Base Unit of Measure a mandatory field, or not? There are tons of other questions to be asked, a question a field, actually. Are these fields mandatory? Yes, and no. Yes for those who really need them, no for those who don’t. And it doesn’t have to be two different companies at all, it can be within a single company that a certain field should better be defined for some operations, while there is no need to have any information in this field for other operations.

So, how do you go about this problem? If you simply decide not to allow users to continue defining a piece of information (a customer, an item, an invoice, etc.) without entering information in all fields that are “mandatory”, you introduce the possibility to have users simply put something there just to be able to continue. That’s baaad.

Another possibility is to have the system not really care about all these fields. So, your users can go, enter information about twenty customers, without entering a value in just about any field there is. And Microsoft Dynamics NAV does exactly this. You enter as much information as you know, but leave the rest for someone else. This is about sharing responsibility. In a proper ERP system, not a single user is ever responsible for all aspects of any entity. While a salesperson is responsible about having correct customer address and contact information, they are not responsible for having posting groups specified for their customers. Accounting department has to define that. There are so many single pieces of information on any master data card in NAV that it is insane to expect that any one single person must know what exactly must be specified, and where. That’s why you have separate departments in your company, and different people are responsible for different things. If you divide the responsibility like that, and person entering information in, say, customer card, doesn’t know what to put in Gen. Bus. Posting Group field, they can simply ignore that field, and go on leaving it empty. The system is pretty happy with that. If the system could live without knowing anything about that customer, it must be able to live without knowing this posting group, right?

But then someone creating a new invoice for such a customer will get an error that says that there is no Gen. Bus. Posting Group defined for this customer! Isn’t that bad? Not only it is not bad, it is desirable. Of course you can’t post anything about a customer without knowing his general business posting group, but it is better to know that one must be specified for a customer at the point you try to post something, rather than when you are defining that customer. Why? Because person entering an invoice is much more likely to know the correct general business posting group for a customer, than the person entering the customer into the system.

So it is all responsibility. But by forcing people to enter information they don’t know anything about, you create an illusion that you are introducing responsibility, when in fact you are introducing irresponsibility. When under pressure of their normal duties, people will simply start putting junk into the system to be able to go on, because otherwise they will have to lose time figuring out all those unknowns, and they will either enter just anything to be able to go on, or they will ask someone else, get an answer, then generalize their own conclusion about how to define that information in the future. Both of these are bad.

It’s not just about responsibility. It is about shared responsibility. It’s better to have more people sharing responsibility, because more people generally know more than one single person does. Microsoft Dynamics NAV helps you with this, because it doesn’t force anyone to enter some information they don’t understand. It simply accepts partially defined information, and will start asking questions only when undefined information is missing, but not earlier.

What to do if your customer is not happy with this? There are several approaches. First, try to educate them, explain what benefit does an approach like this bring, and what pitfalls does the opposite have. Try to point out the problems that may arise from users having to enter information they don’t know. Try to explain how difficult, and expensive (customers are afraid of the word expensive, it is wise to use it in situations like this) it is to unpost incorrect postings, that are incorrect due to bad setup.

If nothing else works, try to implement a solution that doesn’t break the system down. The solution I suggested in my last post is good, because it doesn’t break the concept of shared responsibility. By having everything blocked by default, you don’t force users to enter everything. They can enter as much as they know, then leave all else as is. When someone who really needs that information get an error message saying it is blocked, they can go on and try unblocking it. It is far more likely that person will know what to do with missing information, than the person initially defining it, because missing information is usually related to the transaction which is being entered.

Please, don’t do anything like stopping forms from closing unless everything is entered there. It won’t solve the problem, because there might be five different forms, and as soon you modify all five of them to contain this logic, a sixth one might be underway, or already deployed. Doing things like this introduce inconsistency, because information can be entered into the system using whole bunch of ways other than through forms. There are dataports, there XMLPorts, there are all sorts of outside tools such as SQL Server Integration Services. By putting some logic onto forms, you make users feel everything is OK, while it is not. Not to mention that you are going to have a lot of endless fun trying to really make this work with forms, because forms in NAV are trickier than they seem.

In the end of the day, it is much easier, and less expensive, for customers to simply get used to how NAV works, rather than trying to make it into something it is not.