In a recent webinar, speaker Ying Ki Kwong discussed the risks involved with the management of outsourced projects. Due to time constraints, he and host Phil Lew couldn’t get to every single question asked by attendees. In this follow-up post, they dive deeper into the world of enterprise systems. For more context or to watch the original webinar, please view the slides and presentation recording.
Question 1: How would one handle Agile in contracting?
Ying Ki and Phil’s response: This is a big topic and, unfortunately, no simple answer. Besides the discussion provided in the webinar around Slide 22, please see Section 5 Contractual Considerations in our PNSQC 2018 paper, which can be downloaded here: /archives/wp-content/uploads/2018/06/Kwong_LewPNSQC_2018_paper.pdf. The high level response is: the acquiring organization needs to make sure that the contract provides for interim or milestone deliverables that are reviewable by the organization’s staff or a third-party (e.g. an independent quality assurance contractor); with well defined input (in the form of known requirements and designs) and well defined output (in the form of testable codes or software releases). Precise nature of the statement of work (SOW) and related contractual terms & conditions require careful coordination between a project’s management, procurement, and legal (i.e. Department of Justice in the case of a state); given a level of understanding of the specific life cycle and technical approach to be utilized by the prospective implementation contractor.
Question 2: Should each iteration or sprint require project management sign-off?
Answer: This question may be viewed as related to Question 1. If iterations or sprints are weekly or frequent, it may be impractical to have formal contract deliverables to be reviewed/accepted for every iteration/sprint.
The pragmatic approach in achieving a level of “trust but verify” mindset, management accountability, good quality & risk management, and timely corrective actions would be to set up contract review milestones or contractually required deliverables every few iterations/sprints.
For example, a set of 4 weekly sprints and associated software releases can be lumped into 4-week interim or milestone deliverables; with specific deliverable expectations set forth in a pre-agreed Deliverable Expectation Document (DED) or in the contract itself if formal contract deliverables are specified. See Section 6 Business Aware Testing in our paper identified in the response to Question 1 and discussion provided in the webinar around Slide 25.
Question 3: What about penalty clauses in the contract?
Answer: This question also does not have a simple response. I’m going to quote a paragraph from our paper identified in the response to Question 1: “Terms and conditions in the contract that awards performance (such as early completion) could be a useful incentive for the contractor; as are clauses that delay, retain, or otherwise reduce payment due to non-performance (such as late completion). Especially important would be payment terms (e.g. Net 30 upon invoice), provision for progress payment for deliverables work in progress, retainage (e.g. 15% holdback on all accepted deliverables invoiced until after User Acceptance Test (UAT) and formal acceptance of the system). Use of penalty clauses requires careful coordination between project management, procurement, and legal (i.e. Department of Justice in the case of a state). This makes the UAT a critical part of any contract.”
Also, see the discussion provided in the webinar around Slide 30.
Question 4: Does the use of Agile necessarily align with an enterprise’s governance model? Specifically, for the state of Oregon projects, can Agile be used given requirements around the State’s Stage Gate Review Process?
Answer: The State’s IT governance and the Stage Gate Review Process call for review gates for:
- Stage 1 Initiation,
- Stage 2 Resource & Solution Analysis & Planning,
- Stage 3 Implementation Planning, and
- Stage 4 Execution.
The State of Oregon has more here on the strategy of using the Stage Gates to ensure reviews at the end of each project stage.
In the context of this webinar, the use of Agile is done by the prime contractor during Stage 4 Execution. Thus, the State’s Stage Gates related requirements are largely agnostic to contractor choice of system development life cycle (SDLC) — whether Waterfall, Iterative / Agile or a hybrid thereof.
This is consistent with the State’s procurement philosophy of emphasizing work products and outcomes (functional requirements) over specific life cycle methodology and technical approaches (which are non-functional requirements).
Other states and private sector organizations may have a much greater emphasis on life cycle methodology, technical approach, choice of technology, and enterprise architecture or related standards. In that case, alignment of Agile with an enterprise’s governance model is not a given and requires teamwork between an organization’s IT management, procurement management, and legal (i.e. Department of Justice in the case of a state) in order to prevent mismatch of expectations between stakeholders across the enterprise.
To learn more about integrating risk management and agile processes with enterprise systems, consider attending PNSQC 2019. Thanks so much Ying Ki and Phil! If you have any further questions for either of them, contact Phil Lew via Twitter at @PhilipLew.
Speakers
Ying Ki Kwong is the Statewide Quality Assurance Program Manager in the Office of the State CIO in Oregon state government. Prior to this role, he was IT Investment Oversight Coordinator in the same office and was Project Office Manager of the Medicaid Management Information System Project in the Oregon Department of Human Services. In the private sector, Ying was CEO of an internet B2B portal for trading commodities futures and metals. He was a program manager in the Video & Networking Division of Tektronix, responsible for worldwide applications & channels marketing for a line of video servers in broadcast television applications.
Philip Lew is the Program Chair for PNSQC and the CEO at XBOSoft, a firm specializing in software QA and software testing services. As a Corporate Executive, Development Manager, Product Manager, and Software Engineer, Philip has managed teams to tackle broken processes, develop solutions to difficult problems, and coached others be leaders, managers, and experts. He leverages his academic background combined with hands-on work experience to work with clients and colleagues around the world. For his hobbies, he rides a bicycle and travels the world to quest his thirst for exploration and learning.
