Dr. Rob Hasker SE 3800 NOTE 12 QUALITY Classic Quality Assurance Ensure follow process Solid, reviewed requirements Reviewed design Reviewed, passing tests
Why doesnt we did a good job work? Why isnt this model needed for Scrum? Why do we need something? A little history Whats the Hippocratic oath? Therac-25: medical linear accelerator Generates high-energy beams Targets tumors from multiple angles
June, 1985 61-year old woman receives radiation therapy She received 15,000 to 20,000 rads Typical therapeutic does: in range of 200 rads 1000 rads can be fatal Her breast had to be removed, constant pain in arms Nov 3, 1985: patient dies after receiving 13,000 to 17,000 rads A little history
March, April 1986 Patient receives 16,500 to 25,000 rads in < 1 sec Within weeks: paralysis in an arm, legs, vocal cords Died 5 months later At least 3 other documented deaths Cause: poor software design Interface assumed operators would type slowly Experienced operators could type faster than SW allowed, so data entry was setting a different field than
shown on screen Classic race condition: timing assumptions gone wrong What is the root cause? Simple root cause analysis: ask why 6 times Multiple failures beyond design Manufacturer: poor safety model Relied on hardware reliability Reliability: likely to work safety: no harm Hardware engineers: assumed SW worked
Inadequate logs SW Developers: poor specs, no process Medical authorities: slow to respond But that cant happen today, right? July, 2015: Wired reports ability to remotely control Jeeps Set radio blaring Engaged windshield wipers Disabled accelerator Disabled brakes
Method: Connect via cellular Rewrite entertainment firmware Connect to CAN bus Send signals to engine, brakes, etc. Software Quality Assurance Need a process to ensure SW well designed, tested Who signs off on that in an agile model? What do they look for?
Open questions!! DO-178C Software Considerations in Airborn Systems and Equipment Certification Used by FAA, EASA (European), Transport Canada Published 2012, replaced DO-178B Core: documentation standards Plan for Software Aspects of Certification
Software Quality Assurance Plan Software Development Plan Software Verification Plan Software Levels: failure conditions Level A: Catastrophic Multiple fatalities, often loss of aircraft Level B: Hazardous Crew under high stress, workload Potential fatalities among passengers
Level C: Major Reduced safety margin, slight increase to crew workload Possible passenger discomfort and even injuries Level D: Minor Slight crew function impairment, routine change in flight plan Level E: No safety effect DO 178C
Traceability Requirements Trace required for this certification level SQA Software quality assurance (Wikipedia): Monitoring the SE processes and methods used to ensure quality Sample quality metrics:
Code coverage metrics Errors per line of code during development Defect tracking metrics, esp. defects/user CMMI How can an organization establish it provides quality software? One solution: capability maturity model Another approach What can you accomplish by testing?
Showing the existence of bugs NOT: showing system has no bugs Solution: prove system correct Provide formal specification Basic tool: mathematics esp. set theory Can then prove theorems Limitations Writing specifications is difficult Limited support for theorems
Has been done for compilers, safety critical systems Easy to dismiss, but strong limits to testing Quality past and future Old school software quality Convincing engineers to follow process Ensuring adequate testing CMMI: ensuring quality standards in place, effective Agile: software quality assurance is dead?
Problem: how to certify? Future: proven quality
