| NTIA's Smart Stimulus Review |
| Wednesday, 11 August 2010 15:18 |
Tamia True's proposal to review the results of the NTIA Broadband stimulus is thoughtful and right on target. If delivered to specification, it will be one of the most carefully done studies of any broadband program in the world. The contractor must produce "a report of sufficient methodological, theoretical, and pragmatic rigor to withstand critical scrutiny by technically proficient practitioners, ... Given the nature of the data involved, the Contractor should anticipate employing both quantitative and qualitative methods." I've pasted in below the 31 page statement of work that demands a high standard of analysis.
Broadband policy to date has been shaped by a series of "studies" that fall apart when anyone competent takes a look at them. Nearly all are funded by the companies wanting government money or favors, especially Verizon. They are garbage in, garbage out quality work that have generally been discredited whenever examined by independent scholars. Nearly all are post hoc and some are so bad as to be downright fraudulent.
The evidence has become almost overwhelming that most projects to encourage people to take broadband have almost no effect unless they bring down the price. There's an army of middle class people looking for federal money to explain to poor people how broadband will change their lives and why they should pay the carriers $400-500/year. I've asked nearly all the top officials whether they've seen any of those programs that significantly affect broadband adoption.
The answer is uniformly no. I'd love to be proven wrong about this but success is not in any data publicly available. So I was glad to see "The Final Impact Report shall quantitatively and qualitatively measure the economic and social impact of BTOP grants (including Infrastructure, SBA and PCC)." With $hundreds of millions being spent, if these programs work the results should be clear.
The proposal is clear that that BTOP's #1 priority is "To provide access to broadband service to consumers residing in unserved areas of the country." Especially with the state maps, the number newly offered service is easy to count. NTIA's spreadsheet for a billion dollars of spending only identifies 682 previously unserved homes but there are some more buried in "underserved" categories. The total number will almost certainly be so small that the program has to be judged a failure at its primary goal.
There are two huge problems with this review. The first is that the primary results will not be available until 2014, long after the money has been spent. The officials responsible will probably have moved on. There's no reason that important interim reports couldn't be included.
It's also likely that the terms of the contract will eliminate most, if not all, qualified bidders. No one can bid unless they are qualified under the GSA Mission Oriented Business Integrated Services (MOBIS) Schedule. It takes months, at least, to become MOBIS qualified - long after the contract is awarded. There are only a few dozen people with the experience to do this job well and as far as I can determine none of them works for a MOBIS contractor. Nor have any of the broadband experts I know been contacted. So the odds are the contract will go to one of the usual beltway bandits who have fancy paper credentials and a knowledge of the system but not the subject. They make their living off government work and nearly always deliver a glowing report that praises the agency.
There are half a dozen groups that have the expertise to do this right. Several are university based; a few have developed the expertise because they consult for the industry, including those who did much of the analysis in the broadband plan.
They are probably all excluded by THE SYSTEM. That can't be changed because it's THE SYSTEM. No one is responsible for getting this done right because everyone involved must follow THE SYSTEM.
It's almost enough to turn me into a conservative Republican who believes any government program is likely to fail.
The solicitation is at http://www.aqd.nbc.gov/Business/solic_details.aspx?solid=341. Thanks to the folks at Broadband Breakfast for the pointer to this story.
1 of 31 Attachment #001 Statement of Work For Broadband Technology Opportunities Program (BTOP) Evaluation Study 7/26/2010 12:40 PM 1 Objective The objective of this Statement of Work (SOW) is the production of a methodologically rigorous evaluation of the economic and social impact of the Broadband Technology Opportunities Program (BTOP) grant awards. The purpose of this solicitation is to select a research oriented organization with experience evaluating the impacts of infrastructure investments, which will conduct the appropriate study and provide the Department of Commerce’s National Telecommunications and Information Administration (NTIA) with a comprehensive report of the study results. 2 Background The Department of Commerce’s National Telecommunications and Information Administration (NTIA) is the President’s principal adviser on telecommunications and information policy issues and in this role frequently works with other Executive Branch agencies to develop and present the Administration’s positions on these issues. The American Recovery and Reinvestment Act (ARRA) provided NTIA with $4.7 billion to support the deployment of broadband infrastructure, enhance and expand public computer centers, encourage sustainable adoption of broadband service, and develop and maintain a nationwide public map of broadband service capability and availability. The Recovery Act instructed NTIA to implement the Broadband Technology Opportunities Program to promote five core purposes: 1) To provide access to broadband service to consumers residing in unserved areas of the country; 2) To provide improved access to broadband service to consumers residing in underserved areas of the country; 3) To provide broadband education, awareness, training, access, equipment, and support to: a. schools, libraries, medical and healthcare providers, community colleges and other institutions of higher learning, and other community support 2 of 31 organizations; b. organizations and agencies that provide outreach, access, equipment, and support services to facilitate greater use of broadband services by vulnerable populations (e.g., low-income, unemployed, aged) or c. job-creating strategic facilities located in state- or federally-designated economic development zones; 4) To improve access to, and use of, broadband service by public safety agencies 5) To stimulate the demand for broadband, economic growth, and job creation.1 In facilitating the expansion of broadband infrastructure and adoption of broadband services, NTIA aims to advance the objectives of the Recovery Act to spur job creation and stimulate economic growth and opportunity. Additionally, NTIA seeks to fund projects that will provide long-term economic and social benefits, including improvements to healthcare delivery, education, innovation, and the Nation’s global competitive position.2 NTIA has issued two Notices of Funds Availability (NOFAs). Copies are available at www.ntia.doc.gov/broadbandusa, outlining the objectives, rules, and processes that will govern the award and administration of BTOP grants. As described in these documents in greater detail, BTOP has three types of awards for broadband investment: 1) Infrastructure [including Comprehensive Community Infrastructure (CCI)]: Projects to deploy new or improved broadband Internet facilities (e.g., laying new fiber-optic cables, upgrading wireless towers) to connect households, businesses and “community anchor institutions” such as schools, libraries, hospitals, and public safety facilities. Infrastructure projects funded by BTOP are predominantly middle mile projects, although a small number of last-mile projects were awarded3. CCI projects are middle mile broadband infrastructure projects that offer new or substantially-upgraded service to community anchor institutions. 2) Public Computer Centers (PCC): Projects to establish new public computer facilities or upgrade existing ones that provide broadband access to the general public or to specific vulnerable populations, such as low-income individuals, the unemployed, seniors, children, minorities, and people with disabilities. 1 American Recovery and Reinvestment Act of 2009, Pub. L. No. 111-5, 123 Stat. 115, 512–13 (2009) (Recovery Act). 2 “In the longer-term, BTOP investments will have secondary benefits that will be critical to our Nation’s overall economic future. BTOP-funded projects will help bridge the digital divide improve the Nation’s education, improve access to better health care, heighten safety and security, increase employment options, foster innovation, and boost economic development for communities held back by limited or no access to broadband. These investments also will help preserve America’s economic competitiveness in the world, and, in particular, will accrue benefits to disadvantaged, rural, and remote America.” Testimony of The Honorable Lawrence E. Strickling, Assistant Secretary for Communications and Information, National Telecommunications and Information Administration, U.S. Department of Commerce, Before the Committee on Small Business and Entrepreneurship, U.S. Senate, April 27, 2010, p.12. 3 Executive Office of the President National Economic Council, Recovery Act Investments in Broadband: Leveraging Federal Dollars to Create Jobs and Connect America, December 2009, http://www.whitehouse.gov/sites/default/files/20091217-recovery-act-investments-broadband.pdf 3 of 31 3) Sustainable Broadband Adoption (SBA): Projects to provide a focus on increasing broadband Internet usage and adoption, including projects among vulnerable populations where broadband technology traditionally has been underutilized. Many projects include digital literacy training and outreach campaigns to increase the relevance of broadband in the public’s everyday lives. As of May 2010,4 NTIA has awarded 82 BTOP grants totaling $1.2 billion, with nearly half of the awards applied to Infrastructure projects (predominantly middle mile). NTIA’s priority has been to fund projects that enhance broadband connections to anchor institutions such as schools, libraries, health care facilities, and other entities. NTIA’s Round Two of funding, launched on January 22, 2010, allocated approximately $2.6 billion for grants across PCC, SBA, and CCI project categories. The Recovery Act requires that all BTOP grants be obligated by September 30, 2010 and that BTOP projects be substantially complete within two years of the award date. The Program’s rules require projects to be fully complete within three years of the award date. Data about the grantees’ performance will be made available through quarterly and annual Progress Reports that will be published via www.ntia.doc.gov/broadbandusa. These reports will include, among other things, detailed progress in achieving broadband infrastructure deployment milestones, connections to community anchor institutions, homes, and businesses, public computer workstations made available to the public, and the number of broadband subscribers generated by BTOP’s sustainable broadband adoption projects. In addition, grantees will report information about their use of funds and jobs created, among other requirements, via www.recovery.gov. These data and performance measures will be among the inputs useful in conducting the assessment. 3 Scope of Work The scope of work includes all services related to or necessary for the development of the deliverables as described in Section 10. The scope of work encompasses sufficient Contractor effort to develop a sampling method, assessment criteria, and method of analysis and to use developed tools to conduct a thorough assessment of the benefits that BTOP grant awards are having on broadband availability and adoption, and in achieving economic and social benefits in areas served by the grantees. The scope of work also includes producing a report of sufficient methodological, theoretical, and pragmatic rigor to withstand critical scrutiny by technically proficient practitioners, including an executive summary to be accessible to non-technical readers. The analytical work must be of sufficient quality that it could be accepted by a peer-reviewed academic journal. Given the nature of the data involved, the Contractor should anticipate employing both quantitative and qualitative methods. 4 NTIA Press Release, April 26, 2010, available at: http://www.ntia.doc.gov/press/2010/042610_BTOP_finalRound1.html. 4 of 31 4 Definitions BTOP Broadband Technology Opportunities Program NTIA National Telecommunications and Information Administration Calendar Year The period of time beginning with January 1 and ending with December 31 within the same year. For example, January 1, 2010 through December 31, 2010 is calendar year 2010. Fiscal Year The period of time beginning with October 1 and ending on September 30 used in the federal government to determine its financial year. For example, October 1, 2010 through September 30, 2011 is fiscal year 2011. Working Day Includes the days of the week when official business is conducted, excluding Saturdays, Sundays, Official Holidays and Paid Time Off (including vacation and sick leave.) 5 Specific Tasks The Contractor shall be directly responsible for ensuring the accuracy, timeliness and completion of all tasks under this effort. The effort shall produce several Interim Impact Reports in the form of Case Studies and a Final Impact Report in the form of a longer term Longitudinal Study measuring economic and social impacts. The longitudinal study is the major focus of the effort. In consultation with the Administration, NTIA established rules that allowed up to three years for the period of performance for the broadband grants. Any evaluation of the results of that work would by necessity take longer than 3 years to complete. In Task 1 and 2 the Contractor will lay the ground work setting up the framework, models and methodologies from which the study will be based. In Task 3 the Contractor will execute the study based on the frameworks set up in Task 1 and 2. Task 3 will culminate in two types of reports, Interim Impact Reports and a Final Impact Report. The Interim Reports will be Case Studies providing windows into initial impacts of BTOP awards. These windows will provide ongoing snapshots of the Program’s economic and social impacts and allow for review and possible model adjustments for the longer term in-depth longitudinal study to be conducted. The Final Impact Report shall quantitatively and qualitatively measure the economic and social impact of BTOP grants (including Infrastructure, SBA and PCC). The study should assess whether NTIA’s implementation of BTOP has encouraged the fulfillment of the Recovery Act’s goals described in Section I. In so doing, the Contractor should assess the results of the project to include not only the impact of broadband deployment on the community, but also comparing any impacts to similar communities that did not receive broadband investment funds so as to identify any significant differences and further delineate BTOP project impact. In determining such impacts, the analysis should 5 of 31 include assessments at the individual, household, anchor institution and other unit levels that comprise a given community. 5.1 Task 1: Economic Impact Task 1.1 Economic Impact – Study Criteria Identify the set of economic criteria to be used to perform the study and provide an explanation as to why the selected factors and indicators will support an assessment of BTOP’s short-and long-term economic impacts. The criteria should identify all of the significant economic indicators that need to be evaluated as well as a rationale for the attendant short- and long-term impacts against which defensible research criteria will be established. These may include examining BTOP’s relationship to changes in personal income, consumption, wealth (such as real estate values), employment, productivity, online access to critical functions (health care, education, public safety, telework, and essential government information), competitiveness and entrepreneurship, innovation, and economic growth/output. In ascertaining effects, the evaluators should look at both broadband supply (e.g., access) and demand (e.g., adoption). In addition, the Contractor should examine both direct impacts (e.g., jobs created) and indirect impacts (e.g., multiplier effects to the economy from a new hire’s expenditures). The Contractor should consider that some grant awards will require longer than others to complete and therefore to deliver certain anticipated benefits. Net effects should be identified, e.g., employment that is transplanted from one community without broadband to another with the technology may offset each other rather than yield a positive gain. The Contractor shall also examine telecommunications specific criteria such impacts of funding middle mile versus last mile, impacts of last miles projects and impacts of implemented technologies. Deliverables: (1) Selection of defensible criteria and explanation as to how the selected factors and indicators will support an assessment of BTOP’s economic impact. Task 1.2 Economic Impact – Research Design and Data Research Design Select and provide a justification for the research design(s) chosen to perform the economic impact assessment, including the approaches, methodologies, and models to be used in the analyses. These approaches should include both quantitative and qualitative methods. For example, the methodologies selected should include not only the impact of broadband deployment on the community but also a comparison to similar communities that did not receive broadband investment funds so as to identify significant differences and further delineate potential BTOP project impact. The method(s) of analysis selected should address the statutory purposes and types of BTOP projects as discussed in Section I. 6 of 31 The Contractor should employ a variety of analytical approaches, as appropriate, to identify the benefits conferred by BTOP grants. Such approaches may be used in conjunction and need not be mutually exclusive. Potential approaches include: Econometrics or other statistical methods to measure various macroeconomic indicators, such as employment and output, which certain research studies have shown to be positively related to increases in broadband access and adoption; Input-Output analysis (with our without econometrics) to capture various spending and multiplier effects related to the grants; Cross-sectional analysis to measure impacts at a point in time; Longitudinal analysis over a number of years; Return on Investment (ROI) analysis, including consideration of social opportunity cost of capital, which can be applicable to future public or private investments in broadband. ROI analysis should consider the benefit and or impact in communities that have received BTOP investments against baseline communities that did not receive BTOP funds; Case Studies to identify how the grantee maximized the impact of the BTOP investment; identify successful techniques, tools, materials and strategies used to implement the project; identify any best practices; and gather evidence from third parties, such as consumers and anchor institutions as to the impact of the project in the community. Case studies might explore important concepts that may otherwise be difficult to quantify, such as innovation and entrepreneurship; Delphi Method -- expert-consensus forecasts that allow a best guess today at the eventual results of research outcomes -- to create impact scenarios from BTOP investments and engage expert stakeholders in BTOP initiatives. Scientifically-designed surveys, where necessary, and any associated interviews to help ascertain the main benefits of each grant award. The Contractor shall at a minimum include case studies and a longitudinal analysis in the research design. The Contractor is encouraged to be innovative and forward thinking. The above list is not meant to be exhaustive or limiting. The Contractor may propose an approach(es) not previously mentioned, to employ in conjunction with the case studies and longitudinal analysis. Deliverables: (1) A justification for the research design chosen to perform the assessment, including a detailed explanation of the approaches, methodologies, and models to be employed. Data The Contractor shall identify the data needed to perform the assessment based upon the selection of the study criteria and the selected analytical methodology. All sources of data to be used must be identified, and can include, but are not limited to: 7 of 31 Data from the grant awardees’ initial applications, other data collected by BTOP post-award activities, (e.g., ARRA reporting, BTOP Programmatic Quarterly and Annual Reports); Federal Communications Commission (FCC) and NTIA Broadband Mapping data; and Other sources, such as government and/or private sector sources. The Contractor must also ascertain whether such data will have to be purchased, whether surveys or interviews will be conducted to collect data, and what data may be readily available for inclusion in the selected model(s). The Contractor may also need to address any issues relating to acquiring or using the data, such as privacy concerns, whether a communications plan will have to be developed to acquire primary data, and Paperwork Reduction Act requirements. Further, the Contractor should describe how the analysis will be performed and verified, including how data inaccuracies or gaps will be managed, and what quality assurance processes will be employed. The Contractor should at all times use the most up-todate, accurate and defensible data available. Deliverables: 1) Explanation of what data will be used to conduct the study, including how data inaccuracies or gaps will be managed, and what quality assurance processes will be employed. Task 1.3 Economic Impact – Sampling Method and Approach The Contractor shall address whether every award or a subset of awards will be used for the study and address the sampling method(s) and approach(es). The sampling approach should include an explanation of how to select a stratified sample of awards based upon geographical location, socioeconomic group, award size, or other criteria used to assess BTOP’s effectiveness. The total award pool is estimated to be approximately 250 grants, including roughly 140 Infrastructure projects, 50 SBA projects, and 60 PCC projects. The Contractor shall identify the sampling pool to be used for the Case Studies and for the Longitudinal Study. The sampling pools may or may not intersect. Deliverables: (1) Sampling method and approach. 5.2 Task 2: Social Impact Task 2.1 Social Impact – Study Criteria Develop or identify the set of social impact criteria to be used to perform the study and provide an explanation as to why the selected factors and indicators will support assessing BTOP’s short- and long-term social impacts. 8 of 31 The focus in developing defensive criteria for analysis should be on potential changes in civic participation, healthcare outcomes, educational opportunities, quality of life or standard of living, and other social goods resulting from BTOP investments. It is anticipated that these criteria may be qualitative as well as quantitative. In ascertaining effects, the evaluators should look at both broadband supply (e.g., access) and demand (e.g., adoption). The Contractor should consider again that some grant awards will require longer than others to complete and therefore manifest certain potential benefits. The Contractor shall also examine telecommunications specific criteria such impacts of funding middle mile versus last mile, impacts of last miles projects and impacts of implemented technologies. Deliverables: (1) Selection of defensible criteria and an explanation as to how the selected factors and indicators will support an assessment of BTOP’s social impacts. Task 2.2 Social Impact – Methodology and Data Methodology Select and provide a justification for the research design(s) chosen to perform the social impact assessment, including the approaches, methodologies, and models to be used in the analyses. These approaches should include both quantitative and qualitative methods, with the selection possibly dependent upon award type or other characteristics. For example, the methodology selected could include not only the impact of broadband deployment on the community, but also a comparison to similar communities that did not receive broadband investment funds so as to identify significant differences and thereby further delineate potential BTOP project impact. The method(s) of analysis selected should address the statutory purposes and types of BTOP projects as described in Section 1. The Contractor should employ a variety of analytical approaches, as appropriate, to identify the benefits conferred by BTOP grants. Such approaches may be used in conjunction and need not be mutually exclusive. Potential approaches include: Econometrics or other statistical methods to measure various macroeconomic indicators, such as employment and output, which certain research studies have shown to be positively related to increases in broadband access and adoption; Input-Output analysis (with our without econometrics) to capture various spending and multiplier effects related to the grants; Cross-sectional analysis to measure impacts at a point in time; Longitudinal analysis over a number of years; Return on Investment (ROI) analysis, including consideration of social opportunity cost of capital, which can be applicable to future public or private investments in broadband. ROI analysis should consider the impact in communities that have received BTOP investments against baseline communities that did not receive BTOP funds; 9 of 31 Case Studies to identify how the grantee maximized the impact of the BTOP investment; identify successful techniques, tools, materials and strategies used to implement the project; identify any best practices; and gather evidence from third parties, such as consumers and anchor institutions as to the impact of the project in the community. Case studies might explore important concepts that may otherwise be difficult to quantify, such as innovation and entrepreneurship; Delphi Method -- expert-consensus forecasts that allow a best guess today at the eventual results of research outcomes -- to create impact scenarios from BTOP investments and engage expert stakeholders in BTOP initiatives. Scientifically-designed surveys, where necessary, and any associated interviews to help ascertain the main benefits of each grant award. The Contractor shall at a minimum include case studies and a longitudinal analysis in the research design. The Contractor is encouraged to be innovative and forward thinking. The above list is not meant to be exhaustive or limiting. The Contractor may propose an approach(es) not previously mentioned, to employ in conjunction with the case studies and longitudinal analysis. Deliverables: (1) A justification for the research design chosen to perform the assessment, including approaches, methodologies, and models to be used. Data The Contractor should identify the data needed to perform the assessment based upon the selection of the study criteria and the selected analytical methodology. All sources of data to be used must be identified, and can include, but are not limited to: Data from the grant awardees’ initial applications, other data collected by BTOP post-award activities, (e.g., ARRA reporting, BTOP Programmatic Quarterly and Annual Reports), and Other sources, such as government and/or private sector sources. The Contractor must also ascertain whether such data will have to be purchased, whether surveys or interviews will be conducted to collect data, what data may be already included in the selected model(s). The Contractor may also need to address any issues relating to acquiring or using the data, such as privacy concerns, whether a communications plan will have to be developed to acquire primary data, and Paperwork Reduction Act requirements. Further, the Contractor should describe how the analysis will be performed and verified, including how data inaccuracies or gaps will be managed, and what quality assurance processes will be employed. The Contractor should at all times use the most up-to-date and accurate data available. 10 of 31 Deliverables: 1) Explanation of what data will be used to perform the study, including how data inaccuracies or gaps will be managed, and what quality assurance processes will be employed. Task 2.3 Social Impact – Sampling Method and Approach The Contractor shall address whether every award or a subset of awards will be used for the study and address the sampling method(s) and approach(es). The sampling approach should include an explanation of how to select a stratified sample of awards based upon geographical location, socioeconomic group, award size, or other criteria used to assess BTOP’s effectiveness. The total award pool is estimated to be approximately 250 awards total, including 140 Infrastructure projects, 50 SBA projects, and 60 PCC projects. The Contractor shall identify the sampling pool to be used for the Case Studies and for the Longitudinal Study. The sampling pools may or may not intersect. Deliverables: (1) Sampling method and approach. 5.3 Task 3 – Assessment and Analysis Using the economic and social study criteria, research design and sampling pools defined in Tasks 1 and 2 respectively, the Contractor shall conduct the research study and determine the economic and social impacts of the broadband investments in achieving short- and long-term benefits, both quantitative and qualitative. The assessment shall include conducting surveys and interviews, analyzing grantee quarterly and annual Progress Reports, and incorporating other sources of data identified by the contractor’s sampling methodology in Tasks 1 and 2. The Contractor shall assess the results, to include not only the impact of broadband deployment on the community, but also comparing any impacts to similar communities that did not receive broadband investment funds so as to identify any significant differences and further delineate BTOP project impact. Interim Impact Reports Interim Case Studies PCC/SBA Eighteen months after the award date, the Contractor shall present and deliver the first set of Case Studies to NTIA. The Case Studies shall identify how the grantee maximized the impact of the BTOP investment; identify successful techniques, tools, materials and strategies used to implement the project; identify any best practices; and gather evidence from third parties, such as consumers and anchor institutions as to the impact of the project in the community. The Contractor shall propose how many projects to include based on the implementation progress of individual awards. Due to 11 of 31 the nature of the BTOP program and the Infrastructure projects in particular it will be difficult to measure impact on the Infrastructure projects after one year. Therefore the first set of Case Studies will focus exclusively on the PCC and SBA projects. Interim Case Studies Infrastructure Thirty-six months after the award date, the Contractor shall present and deliver the second set of Case Studies to NTIA. The Case Studies provide follow-up to the PCC and SBA Case Studies previously presented at eighteen months. The Case Studies shall also profile a set of Infrastructure projects. The Case Studies shall identify how the grantee maximized the impact of the BTOP investment; identify successful techniques, tools, materials and strategies used to implement the project; identify any best practices; and gather evidence from third parties, such as consumers and anchor institutions as to the impact of the project in the community. The Contractor shall propose how many Infrastructure projects to include based on the implementation progress of individual awards. The Interim Reports will provide detailed information on the baselines against which future analysis will be compared, should present a rigorous and defensible methodology incorporating all data available at the time, and provide an interim analysis of the initial economic and social impacts of BTOP grants. For example, the Contractor should be able to use quarterly reports to ascertain short-term direct job gains and constructionrelated spending. The Interim Report should provide a roadmap for the duration of the engagement, including additional data gathering and analysis to be performed, expected results and outcomes, and any challenges encountered or recommended amendments to the project plan. The Interim Report should present the initial findings in such a manner that can be released to the public and will provide an initial assessment of the impact of BTOP investments on the economic and social well-being of American communities. Final Summary Impact Report At contract conclusion the Contractor shall present the Final Report, using the Case Studies and the Longitudinal Analysis to report on the outcomes of the assessment and present the findings of the economic and social impact of BTOP grants in achieving the purposes outlined herein. The Final Report should incorporate the complete set of data, including quantitative and qualitative inputs, surveys, case studies, models, methodologies, and analyses outlined in Task 1 and 2 to provide a detailed assessment of the social and economic impact of BTOP grants. The Final Report shall contain final follow-up on PCC, SBA and Infrastructure Case Studies previously presented in the Interim Reports. The Final Report shall also contain the final Longitudinal Analysis and final results from the additional research tools chosen by the Contractor in the Research Design (Task 1.2 in Section 5.1 and Task 2.2 in Section 5.2). 12 of 31 Three months prior to contract conclusion the Final Report will be presented in draft form for NTIA consideration. After the review and revision period, the Final Report shall be delivered. The Contractor shall also deliver any disaggregated, underlying data to the extent practicable, which will be made available by the Government for public use and consumption. Deliverables: 1) Interim Impact Reports, 2) Final Impact Report, 3) Raw Data with an explanation of the data formats and access protocols the contractor plans to use in order to deliver the data to NTIA such that it is suitable both for NTIA and public use. 6 Other Performance Requirements 6.1 Security Requirements A security clearance is not required in performance of this effort 6.2 Contractor Interfaces The Contractor and/or his subcontractors may be required as part of the performance of this effort to work with other Contractors working for the Government. Such other Contractors shall not direct this Contractor and/or their subcontractors in any manner. Also, this Contractor and/or their subcontractors shall not direct the work of other Contractors in any manner. The Government shall establish an initial contact between the Contractor and other Contractors and shall participate in an initial meeting at which the conventions for the scheduling and conduct of future meetings/contacts will be established. Any Contracting Officer’s Technical Representatives (COTR) of other efforts shall be included in any establishment of conventions. 6.3 Contractor Disclosure of Information The Contractor shall maintain, transmit, retain in strictest confidence, and prevent the unauthorized duplication, use, and disclosure of information. The Contractor shall provide information only to employees, Contractors, and subcontractors having a need to know such information in the performance of their duties for this project. Information made available to the Contractor by the Government for the performance or administration of this project shall be used only for those purposes and shall not be used in any other way without the express written consent of NTIA. All data received, processed, evaluated, loaded, and/or created as a result 13 of 31 of this delivery order shall remain the sole property of the Government, unless specific written exception is granted by NTIA. These restrictions are consistent with and do not supersede, conflict with, or otherwise alter the employee obligations, rights, or liabilities created by Executive Order No. 12958; section 7211 of title 5, United States Code (governing disclosures to Congress); section 1034 of title 10, United States Code, as amended by the Military Whistleblower Protection Act (governing disclosure to Congress by members of the military); section 2302(b)(8) of title 5, United States Code, as amended by the Whistleblower Protection Act (governing disclosures of illegality, waste, fraud, abuse or public health or safety threats); the Intelligence Identities Protection Act of 1982 (50 U.S.C. 421 et seq.) (governing disclosures that could expose confidential Government agents); and the statutes which protect against disclosure that may compromise the national security, including sections 641, 793, 794, 798, and 952 of title 18, United States Code, and section 4(b) of the Subversive Activities Act of 1950 (50 U.S.C. 783(b)). The definitions, requirements, obligations, rights, sanctions, and liabilities created by said Executive order and listed statutes are incorporated into this agreement and are controlling. Nothing in this Statement of Work shall bar disclosures to Congress or to an authorized official of an executive agency or the Department of Justice that are essential to reporting a substantial violation of law. See Section 620, Div. H. of title 6, Consolidated Appropriations Act, 2005 (Pub. L. No. 108-447). 6.4 Contractor Use of Data Performance of this effort may require the Contractor to access and use data and information proprietary to a Government agency or Government Contractor which is of such a nature that its dissemination or use, other than in performance of this effort, would be adverse to the interests of the Government and/or others. Contractor and/or Contractor personnel shall not divulge or release data or information developed or obtained in performance of this effort, until made public by the Government, except to authorized Government personnel or upon written approval of the Contracting Officer (CO). The Contractor shall not use, disclose, or reproduce proprietary data that bears a restrictive legend, other than as required in the performance of this effort. Nothing herein shall preclude the use of any data independently acquired by the Contractor without such limitations or prohibit an agreement at no cost to the Government between the Contractor and the data owner which provides for greater rights to the Contractor. Contractor shall supply, at no additional cost to the Government, in machine readable format all data collected under this contract that is relied upon, directly or indirectly, in the analysis and report described in the Deliverables section, hereinafter. The government shall retain all intellectual property rights for all in-process and final deliverables. 14 of 31 6.4.1 Use of Data by NTIA The data provided under this contract/order may be used in any manner by the NTIA or other participating agencies covered under this contract/order, including incorporation within any other information product or system of NTIA/participating agencies, and redistribution in whole or part without attribution as incorporated into other NTIA/participating agencies derivative products as long as such derivate products do not have sufficient information to retrieve the original content of the data provided by this contract/order. 6.4.2 Contractor Restrictions on Data All development deliverables and artifacts, including interim work products and survey data, will be owned by the U.S. Government. Materials and all associated rights therein such as interim draft reports, final draft reports, studies, graphics, maps, photographs, data sets prepared pursuant to this contract shall be considered Government property and delivered as such pursuant to FAR 52.227-17. 6.4.3 Other Copyrightable Works All subject data first produced in the performance of this contract, where subject data consists of copyrightable materials that do not fall within the enumerated categories for work for hire, shall become the property of the government. Contractor hereby transfers to the government full legal title and all right, title, and interest in the copyright to all such subject data whether or not such renditions are actually delivered to the government. Except as specified herein, no subject data, first produced in the performance of this agreement may be published or reproduced by Contractor in whole or in part, in any manner or form, without the government’s prior written consent. Contractor agrees that not right at common law or in equity shall be asserted, and no claim to copyright by statue shall be established by Contractor in any such subject data without the government’s prior written consent. Contractor shall secure the government’s legal title and interests in and to all subject data that is produced for Contractor by third parties pursuant to this Agreement. Excluding the subject data which government owns or has already obtained a license for, Contractor hereby grants to the government a royalty-free, non-exclusive, perpetual, and irrevocable license in all copyrighted or copyrightable subject data not first produced, composed or created in the performance of this contract, but which is incorporated in the material furnished under this contract. Such license includes, without limitation, the rights to reproduce publish, transfer, broadcast, transmit, exploit, display, use, sell and / or dispose of such subject data in any manner, and to authorize others to do so. In the event that Contractor does not have the right to grant such a license with respect to any such subject data, Contractor shall immediately notify NTIA of this fact and obtain the government’s prior written permission to incorporate such subject data in the work. Without this notification, the government will be acting in reliance on this contract and will presume that it possesses all necessary rights and is 15 of 31 free to make whatever use of the subject data that the government determines is in its best interests, including assigning the data to a third-party for its administration and distribution when publicly requested by interested parties. The Contractor hereby warrants that the subject data delivered to the government pursuant to this contract does not infringe statutory copyrights or common law literary rights of contractor or others and contains no matter libelous or otherwise unlawful. 6.5 Hours of Work Contractor work hours are not constrained as the work will not be conducted in Government offices. 6.6 Productive Direct Labor Hours The Contractor can only charge the Government for “Productive Direct Labor Hours”. “Productive Direct Labor Hours” are defined as those hours expended by Contractor personnel in performing work under this effort. This does not include sick leave, vacation, Government or Contractor holidays, jury duty, military leave, or any other kind of administrative leave such as acts of God (i.e. hurricanes, snow storms, tornadoes, etc) Presidential funerals or any other unexpected government closures . 6.7 Government Holidays The following Government holidays are normally observed by Government personnel: New Years Day, Martin Luther King’s Birthday, Presidential Inauguration Day (metropolitan DC area only), President’s Day, Memorial Day, Independence Day, Labor Day, Columbus Day, Veteran’s Day, Thanksgiving Day, Christmas Day, and any other day designated by Federal Statute, Executive Order, and/or Presidential Proclamation. Or any other kind of administrative leave such as acts of God (i.e. hurricanes, snow storms, tornadoes, etc) Presidential funerals or any other unexpected government closures. 6.8 Payment for Unauthorized Work No payments will be made for any unauthorized supplies and/or services or for any unauthorized changes to the work specified herein. This includes any services performed by the Contractor of their own volition or at the request of an individual other than a duly appointed Contracting Officer. Only a duly appointed Contracting Officer is authorized to change the specifications, terms, and conditions under this effort. 16 of 31 6.9 Contractor Personnel The Contractor shall be responsible for managing and overseeing the activities of all Contractor personnel, as well as subcontractor efforts used in performance of this effort. The Contractor's management responsibilities shall include all activities necessary to ensure the accomplishment of timely and effective support, performed in accordance with the requirements contained in the statement of work. Resumes submitted for employees assigned to perform under this statement of work shall contain documented experience directly applicable to the functions to be performed. Further, these prior work experiences shall be specific and of sufficient variety and duration that the employee is able to effectively and efficiently perform the functions assigned. 6.9.1 Project Manager The Contractor shall provide a Project Manager to facilitate Government-Contractor communications. The Project Manager shall be the primary technical and managerial interface between the Contractor and Contracting Officer (CO) and the Contracting Officer’s Technical Representative (COTR) located at NTIA offices located in the Commerce Building 1401 Constitution Ave NW, Washington, D.C. The name of this person, and an alternate or alternates, who shall act for the Contractor when the Manager is absent, be designated in writing to the CO. The Project Manager or alternate will have full authority to act for the Contractor on all contract matters relating to daily operations. The Project Manager or alternate must be available during normal duty hours, as specified herein and to meet with government personnel within 24 hours to discuss problems. The Contractor's Project Manager shall meet with the CO/COTR as necessary to maintain satisfactory performance and to resolve other issues pertaining to Government/Contractor procedures. At these meetings, a mutual effort will be made to resolve any and all problems identified. Written minutes of these meetings shall be prepared by the Contractor, signed by the Contractor's designated representative, and furnished to the Government within two (2) workdays of the subject meeting. The Project Manager and alternate or alternates must be able to read, write, speak, and understand English. 6.9.2 Contractor Employees Contractor personnel, when present in Government offices, shall present a neat appearance and be easily recognized as Contractor employees by wearing a Security Identification Badges at all times while on Government premises. When Contractor personnel attend meetings, answer phones, and work in other situations where their 17 of 31 status is not obvious to third parties they must identify themselves as such to avoid creating the impression that they are government employees. 7 Place(s) of Performance The work will be performed at the Contractor's site. 8 Period of Performance The period of performance for this effort is date of award for four years, through the end of fiscal year 2014, September 30, 2014. 9 Deliverables/Deliverables Schedule In fulfillment of this effort, the Contractor shall provide the following deliverables. All deliverables shall be submitted to the COTR, unless otherwise agreed upon. As described below, these deliverables include: Planning & Status Reports/Events: 1) Orientation Briefing 2) Project Plan 3) Economic Impact – Study Criteria, Research Design and Data, Sampling Method and Approach 4) Social Impact – Study Criteria, Research Design and Data, Sampling Method and Approach 5) Quarterly Status Reports Major Research Reports: 1) Interim Reports (at 18 and 36 months) 2) Interim Executive Briefings 3) Final Report (at conclusion of study) 4) Final Executive Briefing 5) Delivery of all Raw Data – This includes all data that created a foundation for later analysis & conclusions, as well as all data that could be utilized by future researchers. Data content level of detail, storage format, word processing and/or image format, and accessibility to multiple computer stations to be determined by the COTR based on program needs. Note: ADDITIONAL RECOVERY ACT REQUIREMENTS The Contractor is required to comply with all Recovery Act requirements including the reporting requirements entitled “REPORTS ON USE OF FUNDS” as 18 of 31 specified within Section 1512 of the American Recovery and Reinvestment Act of 2009. 9.1 Planning and Status Reports 9.1.1 Orientation Briefing Within two weeks of award, the Contractor shall conduct an orientation briefing for the NTIA Senior Management Team and BTOP staff. The Government does not seek an elaborate orientation briefing nor does it expect the Contractor to expend significant resources in preparation for this briefing. The intent of the briefing is to initiate the communication process between the Government and Contractor by introducing key task participants and explaining their roles, reviewing communication ground rules, and assuring a common understanding of subtask requirements and objectives. The Orientation Briefing will be held at NTIA’s offices in the Hoover Commerce Building 1401 Constitution Ave, NW, Washington D.C. The date and time will be mutually agreed upon by both parties. The completion of this briefing will result in the following: a) Introduction of both Contractor and Government personnel performing work under this Task Order. b) The Contractor will demonstrate confirmation of their understanding of the work to be accomplished under this SOW. 9.1.2 Project Plan The Contractor shall prepare a Project Plan describing the technical approach, organizational resources and management controls to be employed to meet the cost, performance and schedule requirements for this effort. The Project Plan shall detail the key activities and milestones, distance learning technology, allocation of staff and other resources necessary to the successful completion of this effort. The COTR shall receive the revised Project Plan in both hard copy and electronic form, Microsoft Word. Based on the Project Plan, the COTR will provide approval to move forward on activities planned. The Contractor shall request prior approval on all activities not included in the plan or any modifications to the plan after approval has been given. 9.1.3 Economic Impact – Study Criteria, Study Methodology and Data, Sampling Method and Approach (Section 5.1) 1) Selection of defensible criteria and explanation as to how the selected factors and indicators will support an assessment of BTOP’s economic impact. 19 of 31 2) A justification for the research design chosen to perform the assessment, including a detailed explanation of the approaches, methodologies, and models to be employed. 3) Explanation of what data will be used to conduct the study, including how data inaccuracies or gaps will be managed, and what quality assurance processes will be employed. 4) Sampling method and approach and detail on the sampling pool selected. 9.1.4 Social Impact - Study Criteria, Study Methodology and Data, Sampling Method and Approach (Section 5.2 1) Selection of defensible criteria and explanation as to how the selected factors and indicators will support an assessment of BTOP’s economic impact. 2) A justification for the research design chosen to perform the assessment, including a detailed explanation of the approaches, methodologies, and models to be employed. 3) Explanation of what data will be used to conduct the study, including how data inaccuracies or gaps will be managed, and what quality assurance processes will be employed. 4) Sampling method and approach and detail on the sampling pool selected. 9.1.5 Quarterly Status Reports The Contractor shall document and report progress toward the completion of each task in a detailed Quarterly Status Report due 10 days after the end of the fiscal quarter. The status report shall include, at a minimum: Progress for the period: detailed progress report of findings, activities and accomplishments during the reporting period, and summary of work accomplished during the reporting period and percent complete. Activities planned for the next reporting period: planned activities, as well as the status of any/all deliverables, including planned delivery date(s) and actual and/or anticipated delivery date(s). Problems encountered: identification of any problems, issues or delays and recommendations as to their resolution, and any corrective action that was taken to correct identified problems. Strategy revisions: recommended changes to include any lessons learned 20 of 31 9.1.6 Recovery Act Reporting Requirements ADDITIONAL RECOVERY ACT REQUIREMENTS The Contractor is required to comply with all Recovery Act requirements including the reporting requirements entitled “REPORTS ON USE OF FUNDS” as specified within Section 1512 of the American Recovery and Reinvestment Act of 2009. SEC. 1512. REPORTS ON USE OF FUNDS. (a) Short Title- This section may be cited as the `Jobs Accountability Act'. (b) Definitions- In this section: (1) RECIPIENT- The term `recipient'-- (A) means any entity that receives recovery funds directly from the Federal Government (including recovery funds received through grant, loan, or contract) other than an individual; and (B) includes a State that receives recovery funds. (2) RECOVERY FUNDS- The term `recovery funds' means any funds that are made available from appropriations made under this Act. (c) Recipient Reports- Not later than 10 days after the end of each calendar quarter, each recipient that received recovery funds from a Federal agency shall submit a report to that agency that contains-- (1) the total amount of recovery funds received from that agency; (2) the amount of recovery funds received that were expended or obligated to projects or activities; and (3) a detailed list of all projects or activities for which recovery funds were expended or obligated, including-- (A) the name of the project or activity; (B) a description of the project or activity; (C) an evaluation of the completion status of the project or activity; (D) an estimate of the number of jobs created and the number of jobs retained by the project or activity; and (E) for infrastructure investments made by State and local governments, the purpose, total cost, and rationale of the agency for funding the infrastructure investment with funds made available under this Act, and name of the person to contact at the agency if there are concerns with the infrastructure investment. (4) Detailed information on any subcontracts or subgrants awarded by the recipient to include the data elements required to comply with the Federal Funding Accountability and Transparency Act of 2006 (Public Law 109-282), allowing aggregate reporting on awards below $25,000 or to individuals, as prescribed by the Director of the Office of Management and Budget. (d) Agency Reports- Not later than 30 days after the end of each calendar quarter, each agency that made recovery funds available to any recipient shall make the information in reports submitted under subsection (c) publicly available by posting the information on a website. 21 of 31 (e) Other Reports- The Congressional Budget Office and the Government Accountability Office shall comment on the information described in subsection (c)(3)(D) for any reports submitted under subsection (c). Such comments shall be due within 45 days after such reports are submitted. (f) COMPLIANCE- Within 180 days of enactment, as a condition of receipt of funds under this Act, Federal agencies shall require any recipient of such funds to provide the information required under subsection (c). (g) GUIDANCE- Federal agencies, in coordination with the Director of the Office of Management and Budget, shall provide for user-friendly means for recipients of covered funds to meet the requirements of this section. (h) REGISTRATION- Funding recipients required to report information per subsection (c)(4) must register with the Central Contractor Registration database or complete other registration requirements as determined by the Director of the Office of Management and Budget. 9.2 Major Research Reports Reports The Contractor shall provide a set of Interim Reports (after eighteen and thirty-six months), a Draft Final and a Final Report at the conclusion of this effort. The reports will evaluate the impacts spurring broadband infrastructure deployment, enhancing public computing and encouraging broadband subscribership are having on broadband availability and adoption, and in achieving economic and social benefits in geographic areas served by the grantees. The reports shall be suitable for peer review, contain a review of the relevant literature, a discussion of the methodologies (e.g. survey or analytical), and data used in the evaluation. The reports should contain an executive summary of the findings and conclusions suitable for non-technical executive policy makers. The Contractor shall provide the Draft Final Report forty-five months after contract award. This will allow the government sufficient time to review and the contractor sufficient time for report revision. The Contractor shall provide the Final Report in electronic format and hard copy (as appropriate) to be designated by the COTR and shall include all raw data collected during the course of this project in machine readable form. In conjunction with the reports, each report shall coincide with an Executive Briefing which includes a presentation to senior NTIA officials and an executive summary that, in simple language, summarizes the report’s conclusions in a manner suitable for dissemination to the public. The interim and final reports shall follow the same outline format detailed below: I. Executive Summary II. Introduction: Purpose and Objectives III. Methodology of the Analyses IV. Presentation of Analytical Results 22 of 31 V. Evaluation of Results With Respect to Objectives VI. Next Steps Unless otherwise specified, the Government will have a maximum of twenty (20) working days from the day the draft report is received to review the document, provide comments back to the Contractor, approve or disapprove the deliverable(s). During this period, NTIA will engage peer reviewers, as well as government personnel, to provide constructive critiques of the draft intended to improve the quality of the final report. These critiques may include, but are not necessarily limited to, areas such as survey techniques, analytical methods, and consistency with current research in the relevant topical area(s). The Contractor will also have a maximum of ten (10) working days from the day comments are received to incorporate all changes and submit the final deliverable to the Government. All days identified are intended to be workdays unless otherwise specified. Raw Data The Contractor is expected to amass a large amount of raw data in the execution of Case Studies and Longitudinal Study. All raw data shall be turned over to NTIA at the conclusion of the effort with an explanation of the data formats and access protocols the contractor plans to use in order to deliver the data to NTIA such that it is suitable both for NTIA and public use. 23 of 31 9.3 Deliverable Table Reference Milestone/Deliverable Responsibility Date 9.1.1 Orientation Briefing Government / Contractor Contract Award (CA)+ 2 weeks 9.1.2 Project Plan Contractor Draft due at Orientation, Final due (Contract Award (CA)+ 4 weeks) 5.1, 5.2, 9.1.3 Economic Impact – Study Criteria, Study Methodology and Data, Sampling Method and Approach Contractor Contract Award (CA)+ 6 months 5.1, 5.2, 9.1.4 Social Impact – Study Criteria, Study Methodology and Data, Sampling Method and Approach Contractor Contract Award (CA)+ 6 months 9.1.5 Quarterly Status Reports Contractor Contract Award (CA) + 1st quarter of fiscal year, each fiscal year thereafter 5.3, 9.2 Interim Reports Set One (PCC/SBA) Contractor Contract Award (CA) + 18 months 5.3, 9.2 Interim Management Review Briefing Contractor Coincides with Interim Report delivery 5.3, 9.2 Interim Reports Set Two (Infrastructure) Contractor Contract Award (CA) + 36 months 5.3, 9.2 Interim Management Review Briefing Contractor Coincides with Interim Report delivery 5.3, 9.2 Draft Final Report Contractor 3 months prior to Effort Expiration 5.3, 9.2 Final Executive Briefing Contractor 1 week prior to Effort Expiration 5.3, 9.2 Final Report Contractor Effort Expiration 5.1, 5.2, 9.2 Raw Data Delivery Contractor Effort Expiration 9.1.5 RECOVERY ACT Deliverables: “REPORTS ON USE OF FUNDS” as specified within Section 1512 of the American Recovery and Reinvestment Act of 2009. Contractor Not later than 10 days after the end of each calendar quarter 24 of 31 9.4 Inspection and Acceptance Criteria Final inspection and acceptance of all work performed, reports and other deliverables will be performed at the place of delivery by the COTR and such subject area experts as may be empanelled for such purpose. 9.5 General Acceptance Criteria General quality measures, as set forth below, will be applied to each work product received from the Contractor under this statement of work. Accuracy - Work Products shall be accurate in presentation, technical content, and adherence to accepted elements of style. Specifically, the report shall be of sufficient methodological, theoretical, and pragmatic rigor to withstand critical scrutiny by academics and technically proficient practitioners while remaining accessible to non-technical readers. Clarity - Work Products shall be clear and concise. Any/All diagrams shall be easy to understand and be relevant to the supporting narrative. Consistency to Requirements - All work products must satisfy the requirements of this statement of work. File Editing - All text, data and multimedia files shall be delivered in open, industry standard, machine readable data formats such that they can be edited by the Government and used by the outside research community. Format - Work Products shall be submitted in machine hard copy (where applicable) and in electronic format on media mutually agreed upon prior to submission. Hard copy formats shall follow any specified Directives or Manuals. Timeliness - Work Products shall be submitted on or before the due date specified in this statement of work or submitted in accordance with a later scheduled date determined by the Government. 10 Quality Assurance The COTR, and such expert reviewers as may be deemed necessary, will review, for completeness, preliminary or draft documentation that the Contractor submits, and may return it to the Contractor for correction. Absence of any comments by the COTR will not relieve the Contractor of the responsibility for complying with the requirements of this work statement. Final approval and acceptance of documentation required herein shall be by letter of approval and acceptance by COTR. The Contractor shall not 25 of 31 construe any letter of acknowledgment of receipt material as a waiver of review, or as an acknowledgment that the material is in conformance with this work statement. Any approval given during preparation of the documentation, or approval for shipment shall not guarantee the final acceptance of the completed documentation. 11 Administrative Considerations 11.1 Correspondence To promote timely and effective administration, correspondence shall be subject to the following procedures: a) Technical correspondence (where technical issues relating to compliance with the requirements herein) shall be addressed to the Contracting Officer’s Technical Representative (COTR). b) Requests for waivers, deviations or modifications to the requirements, terms or conditions of this SOW) shall be addressed to the Contracting Officer with an information copy to the COTR. 11.2 Points of Contact 11.2.1 Contracting Officer’s Technical Representative (COTR) a) The Contracting Officer’s authorized technical representative (COTR), for this effort is as follows: COTR information to be completed at award b) The COTR is the individual within the Program Management function who has overall technical responsibility for this effort. The COTR supports the CO/CA during administration of this effort by: 1) Making final decisions regarding any recommended rejection of deliverables; 2) Providing technical clarification relative to overall workload matters; 3) Providing advice and guidance to the Contractor in the preparation of deliverables and services; 4) Providing acceptance of deliverable products to assure compliance with requirements. c) The COTR also provide technical direction to the Contractor, i.e., shifting work emphasis between areas of work; fills in details, or otherwise serves to accomplish the purposes of this effort. Technical direction shall be within the general statement of work for this effort. The COTR does NOT have the authority to and may NOT issue any technical direction which: 26 of 31 1) Constitutes an assignment of work outside the general scope of this effort; 2) Constitutes a change as defined in the “Changes” clause; 3) In any way causes an increase or decrease in cost or the time required for performance; 4) Changes any of the terms, conditions, or other requirements of this effort; and 5) Suspends or terminates any portion of this effort. d) All technical direction shall be issued in writing by the COTR or will be confirmed by the COTR in writing within 10 calendar days after verbal issuance. A copy of the written direction shall be furnished to the CO and the CA. e) In addition to providing technical direction, the COTR will: 1) Monitor the Contractor’s technical progress, including surveillance and assessment of performance, and recommend to the CO and CA, any changes in the requirement; 2) Assist the Contractor in the resolution of technical problems encountered during performance; and 3) Perform inspection and acceptance or recommendation for rejection of Contractor deliverables and identify deficiencies in delivered items. This does not replace any other quality assurance inspection requirements that are specified elsewhere within this SOW. f) If in the opinion of the Contractor, any instruction or direction issued by the COTR is outside of their specific authority, the Contractor shall not proceed but shall notify the CO in writing within 5 working days after receipt of any instruction or direction, with an informational copy to the CA. 11.2.2 Contracting Officer The Contracting Officer (CO) for this effort is as follows: Tamia True Acquisition Services Directorate, Herndon Office 381 Elden Street Suite 4000 Herndon, VA 20170 Email Tamia.True@aqd.nbc.gov Phone 703-964-4817 Fax 703-964-8440 27 of 31 12 Invoicing The Contractor shall bill no more than quarterly. Invoices must include, as a minimum, the following information for each individual: Name Time Period Covered Productive Direct Labor Hours for the current billing period and cumulative to Date Labor Category(s) Hourly Rate Any Travel or Other Direct Costs (ODCs) incurred (including supporting documentation/receipts for all charges) for the current billing period and cumulative to date Invoices shall be submitted electronically through GovPay - GovWorks Electronic Invoicing System at https://www.govpay.gov . Please direct all GovPay inquires to the GovPay Help Desk at (703) 787-1200 or helpdesk@govpay.gov. Hardcopy invoices shall not be accepted, unless requested by GovWorks or the GovPay Team. NTIA will hold not less than 30% of the contract amount pending submission and acceptance of the final report. 12.1 GovPay Electronic Invoicing Requirements All payment requests must be submitted electronically through GovPay. “Payment request” means any request for contract financing payment or invoice payment by a Contractor. To constitute a proper invoice, the GovPay payment request must conform to the requirements identified in FAR 32.905(b), “Payment Documentation and Process” and FAR 52.232-25, “Prompt Payment (OCT 2008)”. To ensure the timely processing of invoices GovPay uses an automated “workflow” process to route invoices for review, approvals and payment; as required by the “Prompt Payment Act”. Detailed GovPay information for use of GovPay may be obtained on the Internet at www.govpay.gov. This web site includes user manuals, training resources, instructions for registration and contact information for the GovPay help desk for additional support. All users can access reports on the status of their invoices. Supporting documentation shall be attached to the GovPay invoice in the form of “flat files” in American Standard Code for Information Interchange (ASCII) and an Adobe PDF file. There is a 4MB limitation on file size for these attachments, per header or line item. Facsimile, e-mail, and scanned documents are NOT acceptable electronic forms for payment requests. 28 of 31 GovPay uses the Contractor information in the Central Contractor Registration (CCR) database as one of the components for validating Contractor registration. It is the responsibility of the Contractor to submit accurate and current CCR information. Failure to register and maintain CCR information, or if it has expired, been suspended, been deleted, or could not be found, will result in rejection of your invoice. An invoice submitted during the period for which information in the CCR could not be verified must be resubmitted for payment after successfully registering or updating registration in CCR. Contractors are encouraged to review their CCR information to ensure the most current information is available for GovPay. The CCR Assistance Center is available to provide assistance and answer questions. They can be reached at 1-888-227-2423 or on the web at http://www.ccr.gov. 13 Travel The Contractor will be reimbursed for travel to provide support at a Government site or other site as may be specified and approved by the COTR under this effort. All travel shall be approved, by the COTR, prior to commencement of travel. The Contractor shall be reimbursed for actual allowable, allocable, and reasonable travel costs incurred during performance of this effort in accordance with the Federal Travel Regulations currently in effective on date of travel. 14 Other Direct Costs This requirement is planned as a Firm Fixed Price award. There is no provision for Other Direct Costs. 15 Central Contractor Registration (CCR) – New Contracts The Department of the Interior has adopted the Department of Defense’s Central Contractor Registration database as its database for Contractor information. Accordingly, the following requirements apply to this effort. (a) Definitions. As used in this clause -- “Central Contractor Registration (CCR) database” means the primary Department of Defense (DoD) repository for Contractor information required for the conduct of business with DoD. “Data Universal Number System (DUNS) number” means the 9-digit number assigned by Dun and Bradstreet Information Services to identify unique business entities. 29 of 31 “Data Universal Numbering System+4 (DUNS+4) number” means the DUNS number assigned by Dun and Bradstreet plus a 4-digit suffix that may be assigned by a parent (controlling) business concern. This 4-digit suffix may be assigned at the discretion of the parent business concern for such purposes as identifying sub-units or affiliates of the parent business concern. “Registered in the CCR database” means that all mandatory information, including the DUNS number or the DUNS+4, if applicable, is in the CCR database; the DUNS number has been validated; and all edits have been successfully completed. (b) (1) By submission of an offer, the offeror acknowledges the requirement that it must be registered in the CCR database prior to contract award, during performance, and through final payment of any task order resulting from this solicitation, except for awards to foreign vendors for work to be performed outside the United States. (2) The offeror shall provide its DUNS or, if applicable, its DUNS+4 number with its offer, which will be used by the Contracting Officer to verify that the offeror is registered in the CCR database. (3) Lack of registration in the CCR database will make an offeror ineligible for award of a Department of the Interior task order. (4) DoD has established a goal of registering an applicant in the CCR database within 48 hours after receipt of a complete and accurate application via the Internet. However, registration of an applicant submitting an application through a method other than the Internet may take up to 30 days. Therefore, offerors that are not registered should consider applying for registration immediately upon receipt of this solicitation. (c) The Contractor is responsible for the accuracy and completeness of the data within the CCR, and for any liability resulting from the Government’s reliance on inaccurate or incomplete data. To remain registered in the CCR database after the initial registration, the Contractor is required to confirm on an annual basis that its information in the CCR database is accurate and complete. (d) Following the Contractor’s initial registration in CCR and receipt of any Department of the Interior award of a contract, purchase order, delivery order, task order, basic agreement, basic ordering agreement, or blanket purchase agreement, the Contractor must directly notify the Contracting Officer of any of its changed mandatory business data in CCR within three business days of the change. (See the CCR Handbook at www.ccr.gov for the current mandatory registration data fields, or contact the CCR Assistance Center at 888-227-2423 or 616-961-4725.) 30 of 31 (e) Offerors and Contractors may obtain information on registration and annual confirmation requirements via the Internet at http://www.ccr.gov, from the Defense Electronic Business Program Office (Defense e-Business) at contact.ccr@us.pwcglobal.com, from the CCR Assistance Center at 888-227- 2423 or 616-961-4725, from the Defense Logistic Information Service at dlis_support@dlis.dla.mil. (f) Contractors such as some consultants and sole proprietorships that are small firms that would otherwise have no use for a Dun & Bradstreet (D&B) number may use an alternative D&B registration method. If needing a D&B number principally for CCR registration, such a Contractor should call D&B toll-free at 800/546-0024, and clearly state that it is a very small business and simply needs a DUNS number for the purpose of CCR registration. (g) The Office of Management and Budget has instructed all federal agencies to adopt use of the Central Contactor Registration system as a single, governmentwide system for storing contractor procurement and payment information. The CCR was developed by the Department of Defense and has been in use there since 1998. (h) When registering in the CCR, vendors need to be ready to provide the following information about their companies. (Mandatory fields are marked *.) General Information: *Data Universal Numbering System (DUNS) Number CAGE Code (one will be assigned if you do not have one) *Legal Business Name and Doing Business As (DBA) *US Federal TIN Division Name and Number (if registering as part of a larger organization) *Physical Street Address 1 Physical Street Address 2 *City, State, Zip *Country Mailing Address Information *Date Business Started *Fiscal Year End Close Date *Average # of Employees and Annual Revenue Company Security Level (if applicable) Highest Employee security Level Corporate Web Page URL Corporate Information: *Type of Organization (sole proprietor, corporation, tax exempt corporation, government, etc) *Business Types (more detailed categories similar to above) Minority Owned Business Specific Types Goods/Services: 31 of 31 *North American Industry Classification System (NAICS) Codes *Standard Industrial Classification (SIC) Codes Product Service Codes (PSC Codes) - for services Federal Supply Classification Codes (FSC Codes) - for products Financial Information: EFT information - *Financial Institution *ABA Routing Number *Account Number, Type, & Lockbox Number *Authorization Date *Automated Clearing House *Remittance Information *Accounts Receivable contact *Credit Card accepted (yes/no) Point of Contact: *Primary CCR contact person *Alternate Contact Government Business POC Past Performance POC Electronic Business POC Electronic Data Interchange (EDI) EDI Contact Information The CCR web site includes more detailed instructions for each data element and information on how to obtain a DUNS number. You must have a DUNS number before you can register. |
