1.4.31. Quality Assurance Review Program

(1) Quality assurance is a planned, systematic approach designed to provide confidence that programs, products, policies and procedures will conform to established requirements throughout their life cycle. The term “product” is used in this IRM to describe processes and systems developed, produced and acquired by the IRS to carry out mission-critical functions.

(2) Quality assurance identifies unsatisfactory trends and conditions, defects, non-compliance and corrects factors contributing to improved processes and outcomes.

(3) Quality assurance uses a variety of administrative, analytical and practical methods and techniques to enhance the reliability of products and services. The IRS approaches quality assurance by addressing a range of activities described for each functional program and various systematic activities designed to identify effective and ineffective processes.

(4) Several factors have contributed to a significant impact on the federal government’s quality assurance programs, such as:

1. Internal control requirements

2. Advances in technology

3. More sophisticated and complex products and services

4. More stringent requirements for quality control, reliability and security

5. More emphasis on cost of quality and timely delivery

(5) QAR ensures compliance with OMB and the Department of the Treasury policy through a formal objective assessment process. QAR evaluates and documents internal control activities over financial reporting in:

1. Annual financial statements

2. Other significant internal or external financial reports

3. Compliance with laws and regulations related to those financial reports

(1) Federal Managers' Financial Integrity Act (FMFIA) of 1982 Exhibit 1.4.31-1

(2) Standards for Internal Control in the Federal Government (Green Book) GAO-14-704G Exhibit 1.4.31-2

(3) Department of the Treasury Internal Control Program Requirements

(4) Government Auditing Standards (Yellow Book) GAO-17-313SP

(5) Office of Management and Budget (OMB) Circular A-123, Management’s Responsibility for Enterprise Risk Management and Internal Control, Appendix A: Management of Reporting and Data Integrity Risk

(1) This section provides responsibilities for:

1. CFO

2. Statistics of Income (SOI)

3. Business units

(1) Program Reports: The QAR program reports include:

1. MC ESC briefs

2. Deputy Commissioner Business Performance Reviews (BPRs)

3. Review result notifications to the business units on Forms 14750-A, IRS Quality Assurance Review Report, and 14750-B, IRS Quality Assurance Review Notification

4. Quality Assurance Review Program Summary

(2) Program Effectiveness: The QAR program effectiveness is determined by the:

1. Ability to support the Assurance Statement signed by the Commissioner and submitted to the Department of the Treasury on the status of internal controls.

2. Ability to survey potential reviews, if necessary.

3. Continuous program improvements based on best practice discussions with business units, such as obtaining lessons learned via surveys and QAR program enhancements. Program enhancements include Internal Control Managerial Assessment (ICMA) sampling, and developing, updating and automating processes and official QAR forms.

4. QAR program training and increased IRS managerial awareness of internal controls.

(1) All QAR program documentation and reports are stored on the QAR SharePoint site.

(2) Access to the QAR SharePoint site is restricted to QAR analysts and CFO Management.

(1) The following terms and definitions apply to this program:

1. Concurring Official – The approving official that has the authority to respond to proposed recommendations on behalf of the business unit.

2. Continuous monitoring – A process and technology used to identify compliance and risk issues associated with an agency’s financial, enforcement and operational activities.

3. Corrective action plan – A plan to correct or remediate an identified control deficiency or weakness. Contains a detailed description of how management will implement a corrective action plan that includes completion dates and responsible/concurring official name(s).

4. Effective controls – An effective internal control system provides reasonable assurance that policies, processes, tasks, behaviors and other aspects of an organization, facilitate effective and efficient operations. This includes ensuring the quality of internal and external reporting and compliance with applicable laws, procedures and regulations.

5. Executive Listing – An annual listing of senior executives and business unit points of contact responsible for internal controls.

6. Financial Assurance Control Testing (FACT) – The FACT team is responsible for implementing OMB Circular A-123, Appendix A, Management of Reporting and Data Integrity Risk (for more information, see IRM 1.4.3 Resource Guide for Managers, Financial Assurance Control Testing).

7. Finding – A statement of fact resulting from a review or audit. It can describe a deficiency or an opportunity for improvement in the review report or corrective action plan.

8. Government Accountability Office (GAO) – An independent, nonpartisan agency that works for Congress. GAO investigates how the federal government spends taxpayer dollars.

9. Government Accountability Office (GAO) Internal Control Evaluation Tool – This tool assesses the organizational status of internal control and is based on GAO's Standards for Internal Control in the Federal Government.

10. Government Auditing Standards (Yellow Book) GAO-17-313SP – Auditors of government entities and entities that receive government awards use GAO Government Auditing Standards, commonly referred to as generally accepted government auditing standards (GAGAS) or the Yellow Book, to perform their audits and produce their reports. The Yellow Book contains standards for financial audits, attestation engagements, and performance audits as well as specific requirements for individual auditors and audit organizations.

11. Indeterminable controls  – The QAR analyst cannot determine whether there are sufficient controls in place and unable to identify whether the review is effective or ineffective.

12. Ineffective controls – An ineffective internal control system does not provide reasonable assurance that policies, processes, tasks, behaviors or other aspects of an organization, facilitate effective and efficient operations.

13. Internal Control – The first line of defense in safeguarding assets and preventing and detecting deficiencies, errors and fraud. Internal control systems provide reasonable assurance to achieve effectiveness and efficiency of operations, reliability of financial reporting and compliance with applicable laws and regulations.

14. Internal Control Managerial Assessment (ICMA) – An integral part of the IRS’s annual assurance review process and a valuable tool, completed by all business unit heads of office, for determining the effectiveness and efficiency of, and adherence to, IRS internal controls.

15. Internal Control Review (ICR) program – The ICR program provides business units with insight into the effectiveness of their implemented corrective actions for audit recommendations issued by GAO and TIGTA, as well as evaluate critical controls over IRS programs that may be high risk, high priority or high visibility. This independent examination assists the business units when they review and evaluate their internal control processes. (For more information, see IRM 1.4.32 Resource Guide for Managers, Internal Control Review Program)

16. Management Controls Executive Steering Committee (MC ESC) – An internal oversight committee established to oversee management's design, implementation and operation of internal controls, and to ensure that all business units identify, address and correct internal control deficiencies.

17. Material weakness – Specific instance of a failure in a process/system of a control identified during a review and is critical enough to be reported to managers, senior management or executives. A weakness of this type could be one that would significantly impair fulfillment of the program’s mission, goals and objectives; deprive the public of needed services; violate statutory or regulatory requirements; significantly weaken safeguards against waste, fraud, loss, unauthorized use or misappropriation of funds, property or other assets; result in a conflict of interest; and so on. A material weakness is also reported to GAO and could lead to a modified audit opinion if not adequately addressed.

18. Matter for Consideration(MFC) – An observation or finding that does not rise to the level of a formal recommendation.

19. Quality Assurance Review (QAR) program – The program is based on CFO reviews of ongoing program evaluations, managerial, operational, quality assurance and security reviews performed by the business units. The QAR program leverages existing internal control questionnaires and certification instruments to reduce managerial burden, support internal control objectives and continuously improve programs.

20. Random sampling – A sampling method in which each sample has an equal probability of being chosen.

21. Recommendation – A proposal of controls or improvements that addresses the audit/review finding(s), and when implemented, should correct identified issues.

22. Repetitive review – A review conducted in two preceding fiscal year periods.

23. Responsible Analyst/Subject Matter Expert – The person(s) responsible for the review and the overall procedures of the review.

24. Responsible Official – The executive or director of the Responsible Analyst/Subject Matter Expert responsible for the review and who receives the review report(s) and documentation.

25. Risk – An event that may occur and affect the achievement of a business objective.

26. Risk Assessment – An evaluation of potential hazards, threats or opportunities which could affect an organization’s ability to conduct business.

27. Separation of Duties – An administrative control designed to ensure those who initially completed the review are separate and distinct from those who can adjust live data or correct errors.

28. Significant deficiency – A deficiency or combination of deficiencies in internal controls that is less severe than a material weakness, yet important enough to merit attention by managers or senior executives.

29. Standards for Internal Control in the Federal Government (Green Book) GAO-14-704G – Sets internal control standards for federal agencies. An agency uses the Green Book to design, implement, and operate internal controls to achieve its objectives related to operations, reporting, and compliance as required by the FMFIA.

30. Testing (Sampling) – Procedures to determine whether internal control systems are working in accordance with internal control objectives.

31. Treasury Inspector General for Tax Administration (TIGTA) – The organization established under the IRS Restructuring and Reform Act of 1998 to provide independent oversight of IRS activities. TIGTA promotes economy, efficiency, and effectiveness in administering the internal revenue laws. It is also committed to the prevention and detection of fraud, waste, and abuse within the IRS and related entities.

32. Work papers – The records and documentation created while performing the program evaluations, managerial, operational, quality assurance and security reviews.

(1) The following acronyms apply to this program:

(1) IRM 1.4.2, Resource Guide for Managers, Monitoring and Improving Internal Control

(2) IRM 1.4.3, Resource Guide for Managers, Financial Assurance Control Testing

(3) IRM 1.4.32, Resource Guide for Managers, Internal Control Review Program

(1) The MC ESC serves as the Senior Assessment Team (SAT), which governs the internal control program. The SAT includes individuals from all parts of the organization who may affect internal control over financial reporting, including strategic, operational, financial and programmatic aspects.

(2) The MC ESC meets periodically throughout the year, provides a top leadership perspective addressing important cross-functional issues and advises the Commissioner and Deputy Commissioner.

(1) The QAR program managerial assertions component leverages existing internal control questionnaires, surveys and certification instruments to reduce managerial burden. These instruments include:

1. GAO Internal Control Evaluation Tool

2. Internal Control Managerial Assessment

(1) The review component of the QAR program validates the business units’ assessments of the effectiveness of internal controls by sampling a select number of program evaluations, managerial, operational, quality assurance and security reviews which are analyzed annually by the QAR team.

(2) The QAR process is usually completed between January and July of each fiscal year using Form 14750-A, IRS Quality Assurance Review Report, and is based on documentation from:

1. Business unit reviews performed throughout the IRS

2. IRMs, SOPs, desk guides and other authoritative documents

3. Business unit review reports and results

4. Identified enterprise and audit risks

5. GAO, TIGTA, FACT or ICR audit findings and recommendations

(3) The QAR program’s findings and recommendations are documented and presented to:

1. Designated business unit points of contact over internal controls

2. Executive responsible for the reviewed program area

3. MC ESC

4. Selected review concurring official or responsible official

5. Selected review SME/responsible analyst

(1) The policy and guidance component of the QAR program addresses the CFO duties for developing:

1. IRM 1.4.31, Resource Guide for Managers, Quality Assurance Review Program

2. Standard operating procedures (SOPs), Quality Assurance Review Program

3. Desk guides and other authoritative directives

4. Sample selection, risk assessment and rotation schedule

5. Quality Assurance Review Listing

6. QAR training courses

7. Self-Help Online Tutorials (SHOTs) videos:

1. QAR - Who We Are

2. QAR - Selected Review Process Overview

3. QAR - Interview Questions and IDRs

4. QAR - Completing e-Form 14750

5. QAR - Completing Form 14750-E

8. Business unit specialized trainings

9. Program forms (electronic and paper forms in the 14750 series)

10. Documentation including memoranda, findings and recommendations

11. Reference materials and maintenance of SharePoint sites

12. Retention and storage

(1) The Assurance Statement is a certification that represents the IRS Commissioner's informed judgment as to the overall adequacy and effectiveness of internal controls. The Commissioner will provide one of the following:

1. Unmodified Statement of Assurance: an indication that there are no significant weaknesses or lack of substantial compliance reported.  All evidence indicates that the IRS internal controls are effective and operating as intended.

2. Modified Statement of Assurance: an indication that the IRS’s internal controls are generally effective and operating as intended except for one or more material weaknesses or lack of substantial compliance reported.

3. Statement of No Assurance: an indication that there are no processes in place or there are widespread material weaknesses.

(2) The CFO takes the following steps in the annual assurance process:

1. Request that the business units complete and return the GAO Internal Control Evaluation Tool during the months of January and February to evaluate the status of internal controls within the business unit.

2. Conduct A-123 internal control testing of key financial transactions as required by the Department of the Treasury guidance.

3. Request all business units heads of office to complete the ICMA by the deadline, usually July 31.

4. Update the Quality Assurance Review Listing based on business unit submissions of e-Form 14750, IRS Quality Assurance Review Questionnaire, and supplemental spreadsheets with additional review data.

5. Conduct QARs of program evaluations, managerial, operational, quality assurance and security reviews between January and July of each fiscal year.

6. Conduct Internal Control Review (ICR) program assessments of IRS activities, with the primary mission of providing management accurate analysis and objective recommendations.

(1) Risks can occur at any level of the organization. A risk assessment considers, at a minimum, the likelihood and impact of potential events that may occur. Risks are assessed using a combination of qualitative (for example, written descriptions) and quantitative (for example, number scores) methods.

(2) The CFO assigns risk levels as follows:

1. A review will be designated as “Low Risk” if total points are 6 or less.

2. A review will be designated as “Medium Risk” if total points are between 7 and 12.

3. A review will be designated as “High Risk” if total points are 13 or more.

Note: Questions 2b, 5 - 11 and 13 - 14 from e-Form 14750, IRS Quality Assurance Review Questionnaire, are risk determinative questions. Each “Yes” or “No” response to these questions is assigned a point value.

(3) Once all questions are answered and explanations provided, the points are totaled (automatically) and the risk level for the review is determined using the point scale above. Each question’s point value varies depending on its effect on the overall risk exposure for the review.

(1) The e-form 14750 Submission Call Notification attachment requests that business units review their submissions on the e-Form 14750 Submission SharePoint site and:

1. Identify and submit new internal control reviews.

2. Update existing internal control reviews.

3. Identify which reviews should be deleted and submit Form 14750-E, IRS Quality Assurance Review Questionnaire Deletion Request, for each identified review.

Note: CFO requests the business unit submit a separate e-Form 14750, IRS Quality Assurance Review Questionnaire, for each program evaluation, managerial, operational, quality assurance and security review performed.

(2) To aid in planning for future QAR program reviews, business units are encouraged to provide senior leaders with suggestions on QAR that merit consideration for additional reviews. Input by business units is important and will help in designing a QAR plan to best address issues that are important to the IRS.

(1) The ICMA is one of the components of the Assurance Statement signed by the IRS Commissioner and submitted to the Department of the Treasury.

(2) There is a direct correlation between the ICMA and the QAR process. The ICMA addresses questions regarding Internal Controls.

(3) All business units of the IRS are required to review the adequacy of internal controls, identify risks, report fraud and certify that the control environment for their business unit is working as designed through the annual ICMA.

(4) The CFO obtains a listing of IRS business units’ head of office responses to the ICMA from the Assurance review program manager.

(5) A review of the ICMA internal control responses is based on a judgmental sample, which is a non-random sample that is selected based on the opinion of an expert.

(6) Business Units who identify that they perform a review as part of their internal control operations and do not have an e-Form 14750, IRS Quality Assurance Review Questionnaire, on the Quality Assurance Review Listing are notified by email and asked to complete an e-Form 14750 for the identified review.

(7) The ICMA sample is usually selected in the first quarter of the fiscal year.

(8) The Quality Assurance Review Listing is updated when the e-Form 14750, IRS Quality Assurance Review Questionnaire, is received from the selected managers.

(1) Periodically, new IRS initiatives are identified by the MC ESC or the SET (Senior Executive Team) and require new program evaluations, managerial, operational, quality assurance or security reviews.

(2) Identification of these new reviews would be in addition to the e-Form 14750 Submission Call Notification request and the ICMA sampling that resulted in new reviews being submitted. These new reviews may be the result of GAO, TIGTA, ICR or FACT recommendations. These new reviews may also be the result of an increased awareness of a particular program or new initiative.

(3) The CFO will request that the business units’ points of contact for internal controls or the manager responsible for newly identified processes to complete e-Form 14750, IRS Quality Assurance Review Questionnaire.

(4) The CFO will update and post the Quality Assurance Review Listing on the QAR SharePoint site.

(1) The QAR period begins in January and ends in July. The reviews are timed to support the Commissioner’s Assurance Statement submission to the Department of the Treasury.

(2) QARs continuously monitor business unit activities such as processes, procedures and related reports.

(1) The number of program evaluations, managerial, operational, quality assurance and security reviews selected for each fiscal year is based on a combination of the following:

1. SOI uses random sampling to determine the selected reviews per the criteria provided by the CFO.

2. QAR, ICR and FACT reviews that have been completed within the last two years are exempt from SOI sampling.

   Note: Reviews that have open recommendations will require follow-up in the subsequent fiscal year.

(2) Significant Deficiency: Identify areas for improvement and include open items from remediation and action plans.

(3) Independent Audit Findings: Identify open GAO financial reporting audit recommendations and TIGTA reports.

(1) Each business unit executive responsible for internal controls and the business unit POC is notified by email that their business unit is selected for review. The SME/Responsible Official/Responsible Analyst may also be included on the email correspondence.

(2) The review notification:

1. Identifies the business unit’s selected review(s).

2. Designates the review period for the fiscal year.

3. Indicates the point of contact training date(s).

   Note: QAR will attempt to alleviate repetitive reviews, however, strengthening controls and minimizing risks remain the mission of the Office of Internal Controls.

4. Provides references and attachments such as the e-Form 14750, IRS Quality Assurance Review Questionnaire, associated with each selected review and the current QAR Call Notification email. 

(1) To complete Form 14750-A, IRS Quality Assurance Review Report, the QAR analyst validates the following:

1. Supporting documentation of the business unit’s program evaluations, managerial, operational, quality assurance and security reviews.

2. Information contained on the e-Form 14750, IRS Quality Assurance Review Questionnaire.

(2) The QAR analyst will conduct one or more interviews with the SME for the selected review by phone and electronic communication to gain a better understanding of the review.

(3) The QAR analyst will request supporting documents using Form 14750-C, IRS Quality Assurance Review Information Document Request. Documentation from the business unit should be:

1. Sufficient: Provide complete documentation with their review findings, recommendations and results.

2. Accurate: Permit a reader with no knowledge of the review to reach the same conclusion(s).

3. Restricted to documentation that is applicable to the purpose, goals and outcome of the review.

4. Complete: All evidence needs to demonstrate how the reviewer arrived at the conclusion(s) and should provide a basis for determining whether the conclusions are reasonable and accurate.

(4) The QAR analyst uses information provided on the e-Form 14750, IRS Quality Assurance Review Questionnaire, responses from the interview questions and documents received from Form 14750-C, IRS Quality Assurance Review Information Document Request, to determine whether the review controls are effective, ineffective or indeterminable.

(5) The QAR analyst will provide the SME and POC a preliminary Form 14750-A, IRS Quality Assurance Review Report, for review.

(6) The QAR analyst will share findings and any recommendations at a scheduled concurrence meeting conducted by telephone, Teams or email with the business unit point(s) of contact, concurring official and the executive that has the authority to commit resources to implement agreed recommendations.

(7) In the event there are unagreed recommendations, they will be elevated to the QAR Section Chief, IR Director and ACFO for Internal Controls for evaluation and further discussion with elevated levels of the business unit, if necessary.

(1) The CFO stores QAR work papers and related documents on the QAR SharePoint site or other designated storage; for example, shared drives, if applicable.

(2) Review work papers represent all quality assurance materials, including, but not limited to:

1. Business Units’ training material

2. Completed Form 14750 series QAR forms

3. IRM extracts, Standard Operating Procedures (SOPs), desk guides and other authoritative directives

4. Review reports and requested samples provided by the selected business unit

5. Analyst notes

6. Interview question responses

7. Copy of relevant GAO, TIGTA and CFO audit/review reports

   Note: Joint Audit Management Enterprise System (JAMES) is the Department of the Treasury repository for GAO and TIGTA audit reports.

(3) Written materials from the business unit related to the performed review:

1. System documentation: Includes policies and procedures, organization charts, manuals, memorandums, flowcharts and related written materials necessary to describe organizational structure, operating procedures and administrative practices.

2. Review documentation: Shows the type and scope of review, the responsible official, the pertinent dates and facts, what was tested and how, the key findings and the recommended corrective actions. Documentation should be complete, accurate, clear, legible, relevant and neat.

(4) A status summary of the selected QARs is provided to the MC ESC and included in the Assurance Statement.

(1) QAR documentation is considered a temporary federal record. Review records must be closed out at the end of the fiscal year and deleted five years after closure. Examples of records include:

1. Email communications including review notifications

2. Review documentation, checklists and work papers

3. Review findings and recommendations

(1) The QAR organization structure is comprised of a section chief and program analysts.

(2) QAR Section Chief is responsible for:

1. Providing guidance and management support for planning, analyzing and reporting on the effectiveness of internal controls and mitigating risks.

2. Coordinating with internal stakeholders and business units to conduct QARs to support the Assurance Statement signed by the Commissioner and submitted to the Department of the Treasury.

3. Managing the QAR process.

4. Reporting the results of the QARs.

5. Providing program summary to the ACFO for Internal Controls and CFO.

(3) QAR analysts are responsible for:

1. Conducting reviews of business units program evaluations, managerial, operational, quality assurance and security reviews selected via random sampling.

2. Obtaining and analyzing applicable IRMs, Interim Guidance Memoranda, SOPs, Desk Guides, Job Aids, GAO/TIGTA audit reports and other guidance related to selected reviews.

3. Analyzing supporting documents submitted by the business units.

4. Reporting on the effectiveness of internal controls for assigned reviews.

5. Documenting the findings of the review.

6. Proposing recommendations to strengthen controls.

7. Conducting specialized training for business units, for example, annual training for Points of Contact.

8. Obtaining corrective action plans for open recommendations.

9. Tracking the status of open recommendations.

(1) QAR analyst must have, at minimum, the following knowledge, skills and abilities:

1. Working with functional programs, operations and processes.

2. Demonstrating analytical and evaluative methods, assessing program development or execution and improving organizational efficiency and effectiveness.

3. Preparing comprehensive program analyses and evaluations pertaining to the effectiveness of program operations.

4. Planning, coordinating and establishing operating methods and procedures for accomplishment of project mission.

5. Providing guidance for various analytical studies to resolve substantive problems or improve efficiency and effectiveness of operating line or administrative support programs.

6. Preparing effective written and oral communications.

7. Establishing effective interpersonal relationships with internal and external stakeholders.