Now that AI is an acronym that doesn’t really need to be spelled out (artificial intelligence, just in case you’ve been off the planet the last couple of years), we are hearing and understanding more about its various applications and capabilities.
ADVERTISEMENT |
I’m not an early adapter, but I do recall midway in my journalism career joking that I hoped to retire before they stopped printing things. That was no joke, it turns out. So, the more I read about OpenAI’s ChatGPT replacing writers (that is, people like me), the more curious I became. It’s free to try and easy to sign up, so I gave it a go.
Any talented writer, and especially those who are good typists, can bang out reams of copy that’s long on word count but light on content. What I wanted to know is whether AI-generated writing could pass muster with subject-matter experts.
When asked this question, ChatGPT modestly replied:
…
Comments
ChatGPT's prepare for an audit
First, I question point 1. While it is important for an auditor to know the standard I would hope the auditor is already familiar with the standard
While reading ChatGPT's responses I found myself going back to the task posed to it: "...steps an auditor should take to prepare for an ISO 9001 audit."
Unfortunately, the question is ambiguous. Is the auditor external to the company? If so, points 8, 9 and 10 are not part of the preparation for an audit but rather audit activities. If the auditor is internal is he/she conducting an internal audit (8, 9 and 10 are not preparation) or preparing for an external audit (include point these points as well as taking/verifying corrective actions)?
Point 5 says to "conduct a pre-audit" so an organization can "address any issues before the audit" suggests the organization is preparing for certification rather than undergoing an internal or external audit. If that's the case then ChatGPT should make that clear and should add in the corrective action process based on the findings as part of the auditor (more accurately, the organization) preparing for the external audit.
Details
It's important to be as detailed as possible with your questions to ChatGPT, the better the questions the better the answers you will get. I
Mark, A CGPT user would do
Mark, A CGPT user would do well to first understand some basic prompt engineering strategies.This would allow for an interactive dialog w/ the tool that would serve as a useful method for possible gap analysis or simple brainstorming. In my experience, that is where these tools have been useful.
CGPT
There was no mentioning of opening and closing meetings which are required by most certification bodies for 3rd. party audits.
Be specific
When establishing the length of a written piece, it's essential to provide a range instead of an exact number of words. As someone committed to quality, it's crucial to understand that specificity is key in getting desired results. For instance, indicating a word count between 800 to 850 words offers more flexibility and is more likely to be accurately met, compared to merely asking for exactly 800 words.
5.5/10 - improvement required
It's not a bad effort, I suppose. Equivalent in content to what I'd expect to get out of a typical sales manager.
1. Understanding the ISO 9001 standard is a good idea. But it should be done quite a long time before the audit starts or is even planned! You won't find anything in the standard about the scope of the audit - it's largely up to the auditee to decide how they wish to interpret the standard as it applies to their company (a mistake that many Certification Bureau auditors still also make, so perhaps we should forgive ChatGPT as a relative novice). As an aside, I train my internal auditors not to try and commit great swathes of the standard to memory: they will gain a far greater understanding of when something is wrong by observing carefully - see point 8. The upshot is 'look for something that "feels" wrong and then reconcile it with the standard later'.
2. This is 100% poor advice. There is no requirement to have a Quality Manual and there hasn't been since 2015. Even when they existed, the quality of information in a typical Quality Manual was extremely poor compared to - say - 30 minutes informal preliminary chat with the Quality Manager.
3. There is mixed value in reviewing procedures, I would say. Some companies have huge volumes of procedures that number into the hundreds, if not thousands. If you try to review all these up-front then you'll exhaust yourself before day 1 of the audit (plus, if you're like me, you'll get irked over every single spelling error). However, if your audit is focussing on one particular area (design, say) then reviewing the relevant procedures will likely be useful. The opposite is also true: some companies have no procedures at all, and correctly identify that there is no requirement to have them. You can't review what isn't there: and critically, you cannot draw any conclusions about the effectivity of the management system from the abundance or absence of procedures.
4. Reviewing previous audits is the best bit of advice that ChatGPT has given. You will want to look for evidence that previous nonconformities have been closed out, and use your judgement to understand whether observations/OFIs should be escalated into nonconformities. You should also try to avoid repetition of the previous audit if possible.
5. By ChatGPT's own words, it has sunk itself here. If a pre-audit is an optional step, it cannot possibly be 'one of the ten most important things', can it? Anyway, a pre-audit is the work of an auditee, not the auditor. They have some value if you are undergoing NADCAP or a similar exhaustive checklist audit, but a lot of the time a pre-audit struggles to anticipate the auditor's actions. ISO 9001 certificate holders should be doing internal audits anyway.
6. A good runner-up to point 4 in the usefulness stakes. If you don't have an audit plan then you look unprofessional and any observations you make will be diminished accordingly. Also, you run the risk of being led down the garden path by a wily Quality Manager who only takes you to see things that THEY want you to see. There should be a firm pre-agreement between you and the auditee on what sort of processes and activities you are seeking to audit.
7. Yes, I have no quibble with this. Good communication is a courtesy more than anything.
OK - points 8-10 are now clearly into the realms of 'stuff that is clearly not pre-audit preparation'. But I suppose, with a slight tweak to the question, then it could be considered 'informing the audit plan' or 'telling a novice auditor what to expect'.
8. Onsite interviews are important. But not as important as onsite observations. Most people who you interview will have a practiced 'audit script' which has either been drummed into them by the Quality Manager or is cringeworthily formulated in their own head out of a desire to impress. You learn far more about a site by observing the ethos, the atmosphere in work areas, and having casual chats with employees over lunch. It's much easier to home in on nonconformities by looking for things that are being masked, hidden or skated over and using the right questions to probe them. This is looking for a level of interview technique and 'feels' that ChatGPT will struggle understand, though.
9. Yes, obviously reviewing documentation is a good idea. But it's important to have some pre-conceived idea of what you're looking for. The ISO 9001 standard is (fairly) specific on the nature of records that need to be kept, but it has nothing to say about the contents of those records. Make sure you understand specific content that is relevant to legal compliance or the company's own requirements and audit that specifically rather than trying to assess entire documents in one go.
10. If you prepare an audit report before the audit then you're definitely doing it wrong! But, yes, of course an audit report is an important part of the process. I would probably add a few more guidelines for writing a report: ie. it should not just be a list of nonconformances and 'bad things'. It is equally important to recognise areas of exemplary practice.
Some suggestions that I would put in:
a. Prepare a mental script for the opening meeting. Most auditees are very passive to start with and difficult to engage. Plan exactly what you are going to say during the opening meeting that will put them at ease, inform them exactly what the audit will consist of, and explain why you are there
b. (Plan to) collect objective evidence. This is an important part of the interview/observation process, not just to prove to your superiors that you are doing the job, but to inform the your audit report when you write it later. The concept of objective evidence is probably lost slightly on AI which can happily scrape information off the internet whenever it chooses and therefore doesn't really have to rely on memory.
c. (Plan to) speak to top management. This is sort-of dealt with in point 7 but is very important if you want to audit a business in context. The system you are auditing should be a direct channel for the business strategy and top management's vision. One whole section of the standard, and many other clauses are now dependent on top management commitment. An interview with top management early on will put into context what you expect to see elsewhere.
d. Understand the business processes that you will be auditing. This is complementary to, if not a replacement for, step 2. ISO 9001 has evolved from 'say what you do and do what you say' to the management of a system of interconnected processes. The process approach has no end of benefits, from Lean activities to better root-cause analysis. At the very least every business process should explain its intended outputs and customer (whether that customer is internal or external). When you conduct interviews and observations, you should be auditing that activities that give rise to these outputs and asking how the business ensures the outputs are conforming. There are often grey areas (a good example being a production process that defines the output as a 'product' when actually they mean a 'product that is conformant to specification and produced within a pre-defined time-frame') that are a good source of discussion.
e. Understand how the business identifies and manages risk. The explicit wording relating to risk was new to ISO 9001 in 2015, and its inclusion has been widely criticised, mostly by people who think there's only one way to handle risk (rather disturbingly, risk-management professionals being quite a large contingent). But the concept of risk has always been present and its a concept that completely underpins all quality management systems. Writing procedures is an exercise in risk management. Traditional quality activities such as inspection are exercises in risk management. Writing the quality policy is an exercise in risk management. And critically, there is no standard way for all businesses to manage risk: a risk that is unacceptable in one industry may be embraced in another. You cannot audit procedures and activities in context until you understand the localised approach to risk. It is generally not documented very well, so if there is no evidence available, plan to ask questions in interviews with the Quality Manager and top management.
f. Understand the concept of an audit trail and plan for some potential scenarios that might lead to a fruitful trail. Auditing with a rigid checklist approach is poor practice that gives rise to tunnel vision. Understanding and being able to incorporate audit trails is relevant to the auditee and a fruitful source of evidence. One of the best audit practices I ever witnessed was the auditor breaking off from looking at the quality policy in a routine opening meetingg because I was called away to check a material delivery. We went and audited the goods-in process, checked the purchase records and supplier qualification, looked at the logistics of material distribution and the kanban system and - because it was a live material feed, went and audited the finished product along with all the quality inspection. We captured some calibration info and the names of key personnel to check against training records. Some of it wasn't perfect and I held my hand up and said 'this is definitely an area where we need to improve'. The whole thing took 4 hours, but we covered damn near 75% of the standard on the first morning of a 3-day audit. I get the impression that AI is some way from being able to learn this level of insight, which is thankfully why auditors have to remain human for the foreseeable future.
Add new comment