Medical Systems Engineering - a special case?

On being a fish out of water 

Boring people with your interests is brilliant, especially when they’ve no reasonable excuse to run away. Being a singularly un-fascinating person since my kids arrived, I now have only two topics with which to belabour people: fatherhood and Systems Engineering. The first is, as I hope most people realise, not something to be volunteered until requested out of genuine need – walking up to people and telling them how to deal with their offspring’s foibles is unacceptable and, as a father voicing uneducated opinions at other exhausted men, risks being met with bleary-eyed physical violence. No such risk applies to Systems Engineering and I delight in visiting it upon the ears of the slowest-moving member of whatever social group I happen to be in*.

However, I have realised lately that this is not true in all company. Ironically, the place I feel least inclined to talk about Systems Engineering is in any gathering of Systems Engineers. Communication with them is surprisingly difficult and during my recent excursions back into the world of INCOSE and SaRS I initially worried that my Systems capability had somehow deteriorated since I left the aerospace industry. On reflection, however, I realise my discomfort is a consequence of the particular sort of Systems Engineer I am resulting in significant differences between my own day-to-day experience and that of the majority of the profession. Allow me to explain.

Some History 

Going back a few decades, the history of Systems Engineering has long been bound to industries that deal with things on a massive scale. The space, aerospace, defence, nuclear power and latterly automotive industries, for example, all bring together large, interdisciplinary teams to produce finely-honed products performing challenging and complex functions, and this made them fertile ground for the germ of the Systems approach. Search the web for historic advice on any given Systems technique and the skewed, slightly speckled PDF scan of an ancient-looking Courier-font document will be headed with the name of Lockheed Martin, Boeing, Toyota or Ford.

More recently, the techniques have been transferred usefully to less awesome systems, in my case medical devices**, where regulation and the commercial imperative to wrap cutting edge science with manufacturable, supportable engineering with a short time to market make the approach compelling.

And so the majority of Systems Engineers still spend their entire careers in the more traditional big industry organisations where I started out before jumping ship to medical. So what? Why does that make it hard to talk to my colleagues from Southampton, Coventry and the M4 corridor?

La Diffèrence 

The trouble is, Systems has had to change to be usefully applied in medical device development.

Traditional industry tends to employ large numbers of systems engineers. A few sweeping generalisations in the work they encounter:

1. Systems are big – many people, many companies together work on their development,
2. most interfaces to be managed are to other engineering systems,
3. interfaces to human beings occur, of course, but are made reasonably predictable through controlled training of those humans relevant to the specific interface in question (the consumer automotive industry is a notable exception).

This means you need a lot of systems engineers, all talking to each other and very few with a complete picture, to deal with the interfaces. But you can often deal with humans almost as if they were an engineering system – predictable, testable, and able to be represented by proxy by yet more systems engineers. When considering a technical ensemble solution as a single “system”, most of the complexity is at subsystem boundaries with less to worry about at the system boundary. Managing this complexity is a case of agreeing behaviour between engineering peers.

Let's look at the typical domains of interest for a top-level system owner:

The high level of formality and time allowed for precise system and subsystem definition allows engineers at each level to limit their scope and work largely to specifications pre-agreed through peer-to-peer communication.

Medical devices are a little different. Typically:

1. They are not terribly complex internally (although exceptions exist, especially where large amounts of data or surgical robotics are involved) and the staff required to develop each one is small,
2. most medical systems are stand-alone, and don’t really talk to other engineered systems,
3. interfaces to humans and human processes dominate, and the humans in question can be either very unpredictable (in the case of patient-administered devices) or very imaginative (in the case of expert clinicians).

To illustrate, let’s make some dodgy guesses at where things might sit relative to each other:

This adds up to mean small numbers of systems engineers (typically one person on the project has that title, although others may behave similarly – for example I tend to think of the human factors role as part of the Systems function) and lots of trouble with human unknowns.

It also means boundaries between responsibilities become more blurred. Let's look now at the domain of interest for a medical devices Systems specialist:

Much of the complexity is at the system boundary and dealing with it is a case of learning the context and working out how to live within it. This is the job of the Systems function, and no particular person usually exists to adequately represent the context. The engineer must go out to discover it through bookwork and repeated interviews and observation of many similar stakeholders.

Internal complexity is lower and is dealt with less formally, but the difficulty in integrating scientific capability tends to mean the Systems view must go to a deeper level in each subsystem in order to help rapidly debug unintended interactions.

This wider-reaching view makes the Systems role in medical devices very interesting – you genuinely need to understand the whole project in reasonable detail and the finessing of paper analyses is less central to the job (although they remain an important part of it). This is why my job is fun. It is also why trying to cobble a challenging medical device together without the systems view is inadvisable.

So, to relate this back to my initial social troubles, the differences outlined above mean the tools and techniques encountered in medical devices differ significantly from those employed by the other 95% of Systems people. There are plenty of passing similarities but the imbalances above tend to mean that, at the very least, analysis techniques used very carefully in big industry are used much more lightly in medical devices. In addition, aspects such as human factors have developed separately in the medical field and their implementation does not directly correspond to equivalent functions in other industries. All-in-all, the language, mindset and working environment for the two ends of the Systems scale are all very different, and this makes talking about my profession with those who arguably know it best rather awkward.

Vive it!

Having recognised the source of my discomfort, I don’t intend to let it be. While I have happily sat and listened to William Peine, director of R&D at medical leviathan Medtronic, expounding the importance of Systems Engineering in medical devices, I find that the discipline is not well known among smaller companies who have not yet been there and done that. There is still plenty of scope to innovate the application of a systems approach within small, nimble teams, with the promise of reduced risk and accelerated development without requiring excessive formality.

I encourage everyone in my position to overcome personal unease in engaging with the more formal side, with the intention of gaining a deeper appreciation of our differences so that we can better draw on and adapt (presumably lighten) appropriate techniques to solve problems in our increasingly complexity-bound industry.

* It’s a sharp lesson in fight or flight - nobody goes back for their handbag twice.

** Although note that, as connectedness and data exploitation grows, the medical devices field draws us gradually back towards the huge complexity of the historic engineering marvels. The future promises the need to consider the hospital network, the hospital as a whole and finally an entire national health service as a system of systems. For now, we have just a sniff of this and most Systems work is confined to concern with individual devices and their directly-interfacing clinical technologies and processes, plus their production and maintenance.

On Software Localisation (particularly with relevance to medical AI)

After I posted about my experience at the IET's recent Medical Devices + Data: Future-proofing Health & Care event in Cambridge, I was asked a question here on how software localisation fits into the V&V and regulatory workflow for AI (Artificial Intelligence) SaMD (Software as a Medical Device) products. I'm going to assume that we're talking primarily about linguistic and cultural localisation of software UI here, although there are other aspects which I'll touch on. My answer is fairly brief (I'll put it in italics here) but the ensuing mental tangent that follows, on how to deal with localisation when approaching regulatory submission, was interesting enough to put here.

My answer:

"I should say that software localisation was not discussed during the workshop and that Usability as a whole was not a particular focus.

However, my take is that while there appears to be a promising drift toward harmonisation between the various regulatory regimes, I feel that localisation is one of the areas where this is least likely to be fully achieved. Demonstrating usability in the face of linguistic and cultural differences is an area in which I feel it is reasonable for regulators to continue to demand individual treatment, and while standards may be harmonised release and verification cannot both be simplified without making corrective UI changes more difficult. Localisation remains an additional complication to V&V that must be handled by careful configuration control and architecural care.

While I'm primarily talking about localisation of language here, I think this applies just as much to localisation of AI training, if such a thing is necessary."


And now off into the swirling mental vortex... How do you deal with this? What can you do to reduce the burden of localisation when you develop your AI software system?

Avoid language at all costs

 

First, I would always suggest that effort is better directed at coming up with a language-agnostic UI rather than baking language into software. It is worth working to expunge that last text label in favour of an icon and an explanation in the manual (although note that de-novo icons will tend to fall through to usability validation in any case, carrying some risk with them, so try to use internationally-recognised symbols as far as possible).

Of course, sometimes localised software is the solution for the even worse problem of localised hardware - with the falling cost of deployment of rich software UI, software provides a relatively easy path to getting localised UI onto medical devices that might otherwise require physical configuration changes for different markets. If you can't get away with labelling your buttons with symbols, making them soft buttons at least means you only need one mechanical configuration in manufacture.

Architectural partitioning - is your UI really critical?

 

If your UI is too complex to be conveyed without words on the UI itself, a few options should be considered. The first is whether the UI carries the same safety classification as the underlying functionality. In medical mobile apps, for example, a key medical function will often be hidden beneath incidental, patient-visible functionality provided only to keep the patient in touch with the process, inspire confidence and provide a compelling reason to install the app and keep it up to date.

For example, lets imagine a phone app to allow day-to-day AI control of an active implant. Here's an initial architecture:

 
 Here's the app functionality:

1. Transfer measured physiological data from the implant via Bluetooth*,
2. Filter the data,
3. Store the data remotely for viewing by a clinician*,
4. Use AI to determine ideal implant program based on data and settings from clinician,
5. Adjust implant settings via Bluetooth,*
6. Display implant status to the patient,
7. Display notification of up-coming appointments to the patient,
8. Provide easy access to generic advice on the patient's condition.

1-5 of these are probably medically critical, while 6-8 are likely not to be. Immediately, we can partition the application into two separate parts with differing levels of concern. Let's add the non-medical items in blue:

In fact, there does not need to be any link between the medical and non-medical at all within the phone itself - everything can be handled by transaction with remote services. With this separation, it should become easier to maintain the localised part of the phone software with reduced regulatory scrutiny while leaving the critical part well alone, as they can be completely divorced binaries:

In fact, a further question exists over the status of functions 1, 3 and 5, which are likely to be pure data transfer operations and so also attract a lower level of scrutiny than functions 2 and 4. Additionally, putting the AI in the cloud makes it easier to maintain centrally, and to roll out changes with no app updates required provided the implant interface has not changed (which would be difficult since it's stuck inside the patient). So we end up with this:

Nope, can't do that - I still need localised software

 

If none of that helps and your software still requires localisation and regulatory acceptance, my gut feel is that it is better to localise at build time and keep the variants distinct, such that changes to a single language do not muddy the water for global deployment. However, determining how V&V and submission should work is a question for a regulatory expert or something I'd approach a notified body or the FDA to discuss early on while determining the architecture.

If your regulatory representative can agree that retesting of common core functionality does not need to be repeated for linguistically-different binaries, great. If not, you may need to look at the testing burden balanced against localised UI complexity - it might be better to roll all your languages in at build time and configure at run-time, such that you only need to verify the core once but accept that making changes to a single language is going to be mean reverifying the whole thing. If anyone can provide insight on this (or sees anything wrong with anything I've written above), I'd love to hear it. Otherwise, I'll research it further at some point.

Aesthetics and Technical Diagrams

It’s been a productive afternoon at my desk. After a few deprived months, I’ve just bought a license for an all-time favourite tool: the drawing package, Visio. My weapon of choice for… well, drawing stuff.

Quick, flexible diagramming

Systems Engineering, especially in a fast-paced and relatively agile development environment, needs diagrams to make sense of complexity and communicate structure. Visio is my go-to solution for knocking up such things rapidly and then maintaining them for the longer term.

A drawing tool like this lacks the rigidity of full-blown MBSE applications, but for my purposes this is, as often as not, a good thing, since I largely deal with lightly-structured teams who aren’t themselves familiar with specific modelling languages. Visio’s lack of rules makes it useful for a huge variety of situations and allows a certain degree of nuance to be dealt with within limited scope by agreeing conventions on the spot rather than delving into the SysML book*.

So why not use a pencil? Well, I generally find that if something’s worth drawing, the trial-and-error part of getting it all arranged legibly is both necessary and informative – if I could just draw it, fully-formed, straight out of my head, it probably wasn’t that hard to understand in the first place. Pencil and paper is a terrible (or at least wasteful) way of evolving a drawing. A whiteboard is passable, but generally I just end up photographing it at the end and turning it into a Visio diagram anyway.

Getting back into Visio

Anyway, the splash screen shone its attractive shade of blue once more into my wondering eyeholes and I was surprised to experience a palpable relaxation of the shoulders, as on the return of a beloved family member that I haven’t see in a few months or the reestablishment of supply of something highly addictive. My dependence is physiologically measurable, something which I’m not sure I want Microsoft to know.

And so this afternoon was spent drawing some useful diagrams, but not before I’d spent a few minutes adjusting some stuff (metric measurements, turning off the dynamic grid, turning on the developer tab) and then recreating some basic assets that I’ve become separated from. A few little bits and bobs that I use all the time. Here’s a sample:



Not much to look at, but in quickly coming up with representations to bring understanding to a confused design / team / self I find these basic things make a huge difference:

• The first three items are simple blocks that appear in diagrams for various purposes, to which I like to add connection points in particular places.
• The SysML actor stick man is very useful in defining system interfaces even when not using SysML.
• The pair of clinicians are something I originally made from some standard Visio templates when writing surgical usability reports in a previous job – it’s five minutes to recreate these from the bundled office worker shapes, which are wearing collars and ties rather than scrubs**.

You may ask why I bother to do any of this, when surely you can lash something that conveys meaning together from basic shapes pretty quickly. After all, it takes no artistic talent to produce a pencil diagram, so isn’t this gilding the lily? Well, I don’t think so, for the following reasons.

Mild automation – for a quick draw

Primarily, the basic behaviour of these shapes when combined with smart connectors is helpful – I’m used to working with them, connecting them to each other such that they are easy to rearrange without too much hassle, and this helps me to produce clarity in complex things relatively quickly***.

Look and Feel – selling your thought process

Besides this, they help to mask my lack of artistic ability in making things relatively presentable. The ability to produce consistent look and feel is important for conveying professionalism and instilling into your audience the confidence that you are used to working with such representations and not just making it up as you go along. It’s like putting on a tie to talk to people, only far more important.

Beauty – more than skin deep

Finally, I’m a great believer that all-round basic prettiness in diagrams is an indication of a good architecture or a clear thought process, or whatever the material being presented. Elegance and aesthetics in a diagram surely reflect the suitability of the thing being represented to be distilled within a limited number of dimensions, and therefore act as a measure of its underlying simplicity.

The human brain seems somehow wired to innately recognise and (for some of us at least) appreciate this kind of order without even needing to understand the subject matter being shown. If you can step back until the text becomes illegible, look at the colourful boxes and lines and think “that is, somehow, quite pleasing to the eye”, it indicates certain properties have been achieved:

• symmetry
• balance
• consistency of scale
• association by proximity
• consistent partitioning
• encapsulation of complexity
• graduation in changing properties

These properties in the diagram reflect the same properties in the subject.

Furthermore, the act of trying to wrangle a messy diagram into an attractive shape helps to shed light on how the subject is suboptimal. It draws attention to the areas of inconsistency of scale, the over-complicated interface structure, the over-connectedness of disparate components. The moment you step back from a beautifully structured diagram and realise you’ve left off an arrow which needs to go from one side of the thing right to the other, snaking around all sorts of stuff in the middle, your mind is drawn to an important question – is that the right approach, or should we look to get rid of this anomalous connection through improved architecture?

 

Something’s wrong.

I can’t quite put my finger on it...

Anomalies like this can be fine, but they can also indicate trouble ahead, whether it be a need for catering for special cases in test fixtures, sticky maintenance issues, impending DLL hell / linker nightmares, tricky assembly on the production line or manufacturing niggles spread across multiple global sites.

Plotting for success

So, in conclusion, I remain firmly attached to pleasing diagrams. I’m never happier than when my thoughts are ordered sufficiently that I’m prepared to print them on A0 and nail them to a wall for everyone to see, and I encourage you to adopt this mindset as well. Just get somebody else to operate the plotter – installing the correct driver is nigh-on impossible, the paper loading procedure promises inconvenient loss of fingers and nobody can remember where the cartridges are kept.

* I recommend Jon Holt and Simon Perry’s excellent “SysML For Systems Engineering” (3rd edition on the way this year), for when you’ve absolutely, positively got to rigorously define everything in the room.
** An exercise for the reader. It’s useful to know that if you open the template shape's Shapesheet you can remove the lock that prevents you from ungrouping it.
*** Beyond this, it’s generally a good idea to get to know the tool in a little depth, as you can get an awful lot out of it that isn’t immediately available to the casual user. I recommend www.visguy.com – if you can imagine a thing, this site will probably tell you how to do it.