Saturday, 24 November 2018

Medical Systems Engineering - a special case?


On being a fish out of water 


Boring people with your interests is brilliant, especially when they’ve no reasonable excuse to run away. Being a singularly un-fascinating person since my kids arrived, I now have only two topics with which to belabour people: fatherhood and Systems Engineering. The first is, as I hope most people realise, not something to be volunteered until requested out of genuine need – walking up to people and telling them how to deal with their offspring’s foibles is unacceptable and, as a father voicing uneducated opinions at other exhausted men, risks being met with bleary-eyed physical violence. No such risk applies to Systems Engineering and I delight in visiting it upon the ears of the slowest-moving member of whatever social group I happen to be in*.

However, I have realised lately that this is not true in all company. Ironically, the place I feel least inclined to talk about Systems Engineering is in any gathering of Systems Engineers. Communication with them is surprisingly difficult and during my recent excursions back into the world of INCOSE and SaRS I initially worried that my Systems capability had somehow deteriorated since I left the aerospace industry. On reflection, however, I realise my discomfort is a consequence of the particular sort of Systems Engineer I am resulting in significant differences between my own day-to-day experience and that of the majority of the profession. Allow me to explain.

Some History 


Going back a few decades, the history of Systems Engineering has long been bound to industries that deal with things on a massive scale. The space, aerospace, defence, nuclear power and latterly automotive industries, for example, all bring together large, interdisciplinary teams to produce finely-honed products performing challenging and complex functions, and this made them fertile ground for the germ of the Systems approach. Search the web for historic advice on any given Systems technique and the skewed, slightly speckled PDF scan of an ancient-looking Courier-font document will be headed with the name of Lockheed Martin, Boeing, Toyota or Ford.

More recently, the techniques have been transferred usefully to less awesome systems, in my case medical devices**, where regulation and the commercial imperative to wrap cutting edge science with manufacturable, supportable engineering with a short time to market make the approach compelling.

And so the majority of Systems Engineers still spend their entire careers in the more traditional big industry organisations where I started out before jumping ship to medical. So what? Why does that make it hard to talk to my colleagues from Southampton, Coventry and the M4 corridor?

La Diffèrence 


The trouble is, Systems has had to change to be usefully applied in medical device development.

Traditional industry tends to employ large numbers of systems engineers. A few sweeping generalisations in the work they encounter:

  1. Systems are big – many people, many companies together work on their development,
  2. most interfaces to be managed are to other engineering systems,
  3. interfaces to human beings occur, of course, but are made reasonably predictable through controlled training of those humans relevant to the specific interface in question (the consumer automotive industry is a notable exception).
This means you need a lot of systems engineers, all talking to each other and very few with a complete picture, to deal with the interfaces. But you can often deal with humans almost as if they were an engineering system – predictable, testable, and able to be represented by proxy by yet more systems engineers. When considering a technical ensemble solution as a single “system”, most of the complexity is at subsystem boundaries with less to worry about at the system boundary. Managing this complexity is a case of agreeing behaviour between engineering peers.

Let's look at the typical domains of interest for a top-level system owner:

The high level of formality and time allowed for precise system and subsystem definition allows engineers at each level to limit their scope and work largely to specifications pre-agreed through peer-to-peer communication.

Medical devices are a little different. Typically:
  1. They are not terribly complex internally (although exceptions exist, especially where large amounts of data or surgical robotics are involved) and the staff required to develop each one is small,
  2. most medical systems are stand-alone, and don’t really talk to other engineered systems,
  3. interfaces to humans and human processes dominate, and the humans in question can be either very unpredictable (in the case of patient-administered devices) or very imaginative (in the case of expert clinicians).
To illustrate, let’s make some dodgy guesses at where things might sit relative to each other:


This adds up to mean small numbers of systems engineers (typically one person on the project has that title, although others may behave similarly – for example I tend to think of the human factors role as part of the Systems function) and lots of trouble with human unknowns.

It also means boundaries between responsibilities become more blurred. Let's look now at the domain of interest for a medical devices Systems specialist:

Much of the complexity is at the system boundary and dealing with it is a case of learning the context and working out how to live within it. This is the job of the Systems function, and no particular person usually exists to adequately represent the context. The engineer must go out to discover it through bookwork and repeated interviews and observation of many similar stakeholders.

Internal complexity is lower and is dealt with less formally, but the difficulty in integrating scientific capability tends to mean the Systems view must go to a deeper level in each subsystem in order to help rapidly debug unintended interactions.

This wider-reaching view makes the Systems role in medical devices very interesting – you genuinely need to understand the whole project in reasonable detail and the finessing of paper analyses is less central to the job (although they remain an important part of it). This is why my job is fun. It is also why trying to cobble a challenging medical device together without the systems view is inadvisable.

So, to relate this back to my initial social troubles, the differences outlined above mean the tools and techniques encountered in medical devices differ significantly from those employed by the other 95% of Systems people. There are plenty of passing similarities but the imbalances above tend to mean that, at the very least, analysis techniques used very carefully in big industry are used much more lightly in medical devices. In addition, aspects such as human factors have developed separately in the medical field and their implementation does not directly correspond to equivalent functions in other industries. All-in-all, the language, mindset and working environment for the two ends of the Systems scale are all very different, and this makes talking about my profession with those who arguably know it best rather awkward.

Vive it!


Having recognised the source of my discomfort, I don’t intend to let it be. While I have happily sat and listened to William Peine, director of R&D at medical leviathan Medtronic, expounding the importance of Systems Engineering in medical devices, I find that the discipline is not well known among smaller companies who have not yet been there and done that. There is still plenty of scope to innovate the application of a systems approach within small, nimble teams, with the promise of reduced risk and accelerated development without requiring excessive formality.

I encourage everyone in my position to overcome personal unease in engaging with the more formal side, with the intention of gaining a deeper appreciation of our differences so that we can better draw on and adapt (presumably lighten) appropriate techniques to solve problems in our increasingly complexity-bound industry.

* It’s a sharp lesson in fight or flight - nobody goes back for their handbag twice.

** Although note that, as connectedness and data exploitation grows, the medical devices field draws us gradually back towards the huge complexity of the historic engineering marvels. The future promises the need to consider the hospital network, the hospital as a whole and finally an entire national health service as a system of systems. For now, we have just a sniff of this and most Systems work is confined to concern with individual devices and their directly-interfacing clinical technologies and processes, plus their production and maintenance.

On Software Localisation (particularly with relevance to medical AI)


After I posted about my experience at the IET's recent Medical Devices + Data: Future-proofing Health & Care event in Cambridge, I was asked a question here on how software localisation fits into the V&V and regulatory workflow for AI (Artificial Intelligence) SaMD (Software as a Medical Device) products. I'm going to assume that we're talking primarily about linguistic and cultural localisation of software UI here, although there are other aspects which I'll touch on. My answer is fairly brief (I'll put it in italics here) but the ensuing mental tangent that follows, on how to deal with localisation when approaching regulatory submission, was interesting enough to put here.

My answer:

"I should say that software localisation was not discussed during the workshop and that Usability as a whole was not a particular focus.

However, my take is that while there appears to be a promising drift toward harmonisation between the various regulatory regimes, I feel that localisation is one of the areas where this is least likely to be fully achieved. Demonstrating usability in the face of linguistic and cultural differences is an area in which I feel it is reasonable for regulators to continue to demand individual treatment, and while standards may be harmonised release and verification cannot both be simplified without making corrective UI changes more difficult. Localisation remains an additional complication to V&V that must be handled by careful configuration control and architecural care.

While I'm primarily talking about localisation of language here, I think this applies just as much to localisation of AI training, if such a thing is necessary."


And now off into the swirling mental vortex... How do you deal with this? What can you do to reduce the burden of localisation when you develop your AI software system?

Avoid language at all costs

 

First, I would always suggest that effort is better directed at coming up with a language-agnostic UI rather than baking language into software. It is worth working to expunge that last text label in favour of an icon and an explanation in the manual (although note that de-novo icons will tend to fall through to usability validation in any case, carrying some risk with them, so try to use internationally-recognised symbols as far as possible).

Of course, sometimes localised software is the solution for the even worse problem of localised hardware - with the falling cost of deployment of rich software UI, software provides a relatively easy path to getting localised UI onto medical devices that might otherwise require physical configuration changes for different markets. If you can't get away with labelling your buttons with symbols, making them soft buttons at least means you only need one mechanical configuration in manufacture.

Architectural partitioning - is your UI really critical?

 

If your UI is too complex to be conveyed without words on the UI itself, a few options should be considered. The first is whether the UI carries the same safety classification as the underlying functionality. In medical mobile apps, for example, a key medical function will often be hidden beneath incidental, patient-visible functionality provided only to keep the patient in touch with the process, inspire confidence and provide a compelling reason to install the app and keep it up to date.

For example, lets imagine a phone app to allow day-to-day AI control of an active implant. Here's an initial architecture:
 
 Here's the app functionality:

  1. Transfer measured physiological data from the implant via Bluetooth*,
  2. Filter the data,
  3. Store the data remotely for viewing by a clinician*,
  4. Use AI to determine ideal implant program based on data and settings from clinician,
  5. Adjust implant settings via Bluetooth,*
  6. Display implant status to the patient,
  7. Display notification of up-coming appointments to the patient,
  8. Provide easy access to generic advice on the patient's condition.
1-5 of these are probably medically critical, while 6-8 are likely not to be. Immediately, we can partition the application into two separate parts with differing levels of concern. Let's add the non-medical items in blue:

In fact, there does not need to be any link between the medical and non-medical at all within the phone itself - everything can be handled by transaction with remote services. With this separation, it should become easier to maintain the localised part of the phone software with reduced regulatory scrutiny while leaving the critical part well alone, as they can be completely divorced binaries:


In fact, a further question exists over the status of functions 1, 3 and 5, which are likely to be pure data transfer operations and so also attract a lower level of scrutiny than functions 2 and 4. Additionally, putting the AI in the cloud makes it easier to maintain centrally, and to roll out changes with no app updates required provided the implant interface has not changed (which would be difficult since it's stuck inside the patient). So we end up with this:



Nope, can't do that - I still need localised software

 

If none of that helps and your software still requires localisation and regulatory acceptance, my gut feel is that it is better to localise at build time and keep the variants distinct, such that changes to a single language do not muddy the water for global deployment. However, determining how V&V and submission should work is a question for a regulatory expert or something I'd approach a notified body or the FDA to discuss early on while determining the architecture.

If your regulatory representative can agree that retesting of common core functionality does not need to be repeated for linguistically-different binaries, great. If not, you may need to look at the testing burden balanced against localised UI complexity - it might be better to roll all your languages in at build time and configure at run-time, such that you only need to verify the core once but accept that making changes to a single language is going to be mean reverifying the whole thing. If anyone can provide insight on this (or sees anything wrong with anything I've written above), I'd love to hear it. Otherwise, I'll research it further at some point.