Stories about Software


Implied Acceptance Criteria

I’m on a Scrum team these days, serving as Product Owner, and I was watching developers do functionality demos. This has generally been of the form of them walking me through the happy path, and then me taking it for a test drive and trying to put myself in a user, ‘cleverly’ typing “twenty five” into a text box that clearly wants an integer to see what happens. Yellow screen of death? Generic error message? Scornful validation message such as “dude, what’s wrong with you?” Helpful validation message such as “your response for how many children you have cannot be a negative number, a decimal, or typeset nonsense?”

If what happens isn’t what I think should happen, it’s then off to check out the acceptance criteria/tests. Did it say anything in there about a validation message? Should it have to? Are there things that ought to be “universal acceptance criteria?”

To me, the answer to that question is, “sort of, but I’d prefer to think of them more as default/implied ACs.” And so I started to enumerate them on an internal wiki, kind of as an exercise for myself to see if there were certain things that I think should always be true. Maybe these are like the equivalent of code contracts, but contracts between the developer and the user. Here are some ideas that I came up with:

  • When user supplies invalid input, a message should be shown explaining why the input was not accepted.
  • The way to get back to where you just were should always be available and obvious (e.g. “cancel” button or “back” link or something).
  • Nothing that happens should ever result in a “yellow screen” or whatever equivalent indicates to the user that whatever is going wrong is something you never dreamed of (for production release, anyway — I have no issues with crashes like this in internal or beta tests as part of a “fail early” strategy)
  • If an exception occurs that the user would understand, it is explained to the user (e.g. “The connection to the database was lost.”)
  • There is never a point during the normal course of usage where the user thinks, “should I just, like, wait, or is it frozen?”
  • If it’s possible to prevent the user from doing the wrong thing, the user is prevented from doing the wrong thing (e.g. if user clicks “save” and that’s a long running operation, “save” button is disabled until it’s okay to click again)

These are some things that I think ought to be true of your application’s behavior unless there is a good reason to deviate. These are things that I generally think of when I’m working, even if nothing is spelled out explicitly. I think of these and I think of more, depending on the application context (e.g. is this something that should be permission controlled or is this something where the verbiage might be changed later?) Over the years, I’ve developed a pretty lengthy mental checklist for what I consider good application experience, even without consciously realizing that I’ve done so for the most part. But I think it’s important for us to try to grow this evoked set in our minds as we go.

So what about you? Do you have anything you’d add to this list — any glaring omissions? I’m actually looking to tabulate some of these things into a working document and perhaps expand it to cover other kinds of minimal checklists for various scenarios (such as testing your own code via running the application, finding and fixing a bug, etc). Perhaps when I get it more fleshed out at some point, I’ll revisit and post again, but either way, I’d be interested to hear what you consider to be table stakes/minimum standards for how your applications interact with users.

  • Pingback: The Baeldung Weekly Review 16()

  • Geoff Mazeroff

    Having a way to undo operations. Knowing there’s a safety net makes me less apprehensive when using your product/tool.

  • Eric Olsson

    I really like that idea of default or implied acceptance criteria. I also like to think of them as just things that we, as software professionals, should do as a matter of course. You nailed one of them when stating that actions shouldn’t result in an unhandled exception. Forcing the product owner to enumerate that as an acceptance criterion is pedantic and detracts from a good working relationship a team should have with the product owner.
    Limiting the discussion to default/implied ACs when it comes to user interaction, these are some of the ones that I thought of:
    – Focus should move among controls in a reasonable order
    – Ensure text prompts are spelled correctly and should be grammatically parallel
    I am vacillating when it comes to keyboard shortcuts. Application-wide should be specified by the product owner. However, if you are working on a desktop client, you can specify default/cancel properties on the respective buttons. Should that be implied? I can be convinced either way.
    Great post, Erik. I hope that you get some good suggestions.

  • “Default”, or even the dreaded “standard” are fine, but “implied” sets my teeth on edge. Defaults and standards can be communicated and agreed to – unspoken expectations are just an invitation to conflict and avoidable failure.

  • I more mean that they’ve historically been implied. My goal currently is to define them and make them explicit 🙂

  • Makes sense, though I might offer an out for where it makes sense with something like “either offers undo or states when undo is not possible.”

  • As I think this through a bit more, it seems like there could be a sort of consensus hierarchy. For instance, there are the common sense, universal(-ish) things such as not bombing out unexpectedly with no explanation. But then the stuff about, say, keyboard shortcuts, might be part of a project level checklist that the developers and users agree to.

    I’m kind of freewheeling with this. I think my main goal is to lay out a series of things that developers on a project can go through as a sanity and sort of ‘professionalism’ check for the features they deliver. Like, “okay, happy path works, but now if I want to deliver something I can be proud of, I should verify as a matter of course that….”

    And, I think that checking spelling/grammar for prompts is certainly not too much to ask, nor is predictable flow through the GUI.

  • One of the thing that I think is missing is making sure the UI and texts respect the same standard in all the application. All the functionnalities should fit well together and look like they belong to the same application. If there is no reason for two buttons or two messages windows to behave differently or to look different, they should be the same.

  • Great post as always.

    I kinda enforce these types of validations through Definition of Done. It’s then the responsibility of the developer & the code reviewer to make sure these are done (if needed). I wrote my thoughts on this down a while back: http://www.mehdi-khalili.com/definition-of-done-in-an-mvc-project

  • Uniformity/consistency of the UX is definitely a good one to mention. Thanks!

  • I had entertained the idea of incorporating it into the DoD, but my initial thought was that this would be “too much” for that venue and that I’d rather keep the DoD kind of minimal. But as I think on it, I don’t really know why that’s my reaction. I like the list that you’ve compiled, and it makes sense for a team to say “these things all have to be true for it to be considered done.” So, it might require some further deliberation to decide where the line is drawn with Acceptance Criteria versus DoD, but I like your concept.

  • You’re right: a large DoD that causes too much burden is not good. It might be ignored one item at a time & eventually dumped if care is not taken. So that’s a valid concern & I try to keep it rather light to avoid this.

    That said I personally think Acceptance Criteria by definition are bound to user stories and talk about business requirements. Everything else falls under coding & technical standards and should be validated as part of the development process, either as part of DoD or a less formal code review checklist!

  • Darío Díaz Anzalone

    Just a small one on UI: if there is a button whose action can not be done at the moment, it should be disabled, or, if necessary, show a message with why the action can not be done at the moment. I hate buttons that do nothing and you don’t know if you have clicked them or if something is not working…

  • Yeah, I think “don’t have do-nothing buttons” is definitely a good one to have. I wonder if it might be generalized to some kind of “principle of least surprise” concept. Of course, the danger there is that’s pretty subjective and open to interpretation whereas what you’re suggesting is concrete.

  • Thomas Lassanske

    In video game development, we have standards that are developed for each discipline or type of content (art, animation, AI, UI, etc), which are developed during preproduction (research and design phase). While these may change from project to project as hardware and technology changes, for the course of the current project, they become the implied standards for each of the stories written for each discipline and implemented during production. I see these as a second layer of implied standards to your “universal” implied standards.

  • I’m curious about these — can you give some examples of standards for some of the content types?

  • Thomas Lassanske

    Sure! Most of these will obviously be related to performance, as many games are trying to squeeze the most out of the hardware. Others will be related to the design of the game and a desire to provide a consistent experience. Most will be related to both, as they often drive each other.

    For visual content, there will be triangle count targets for different types of models (signature elements get more), and average pixel densities and maximum overall size for the textures. For level design, there may be an acceptable overdraw count, total number of triangles and/or models per camera location, and numbers or coverage of different types of elements, such as translucent and reflective surfaces. Concept and/or Art Direction will drive style parameters of the entire game and individual segments of it (color palette, scale). Animation will likely have cap on maximum compression error allowed and total animation data per model, and there will likely be a minimum set of animations and transitions required for some models.

    Code tends to have fewer of these project-specific implied standards, and the ones documented are often driven by the requirements of the underlying engine technology, or the design of the game. For example, a design that emphasizes the quality of the AI will likely have some AI standards to ensure that goal is met consistently, whereas a graphical masterpiece may not. Some general items might include size of the network footprint, number of milliseconds of processing time allowed per system or per AI (per frame).

  • Thanks for the examples. I’ve never, in my travels as a consultant or an employee, actually worked in that domain at all. It’s interesting to hear and understand what that looks like.