Loading...
Fiber Optics Master Plan - Aug 2020 Fiber Optic Master Plan Prepared for the City of Cupertino, California August 2020 City of Cupertino | Fiber Optic Master Plan | August 2020 ii Contents 1 Executive Summary .............................................................................................................................. 1 1.1 Overview of Existing Infrastructure .................................................................................................... 3 1.2 Needs Assessment ................................................................................................................................. 4 1.3 Use Cases ................................................................................................................................................ 6 1.4 Cost Estimates for Expanding the City’s Fiber (Construction or Dark Fiber Lease) .................. 6 1.5 Business Models and Funding Options for Fiber Expansion ........................................................ 8 1.6 Recommendation for a Dig Once Policy ............................................................................................ 9 2 Overview of Existing City Communications Infrastructure and Related Assets ......................... 10 2.1 Data Related to City’s Infrastructure ............................................................................................. 10 2.2 Fiber Optic Network ............................................................................................................................ 11 2.2.1 Construction Background and Restrictions on Use.............................................................................. 12 2.2.2 Routes and Connections ...................................................................................................................... 13 2.2.3 Operations ........................................................................................................................................... 14 2.2.4 Applications ......................................................................................................................................... 14 2.2.5 Conduit and Handholes ....................................................................................................................... 15 2.2.6 Traffic Communications ....................................................................................................................... 16 2.3 Emergency Operations Center ....................................................................................................... 17 2.4 ARKnet .................................................................................................................................................... 18 2.5 Smart City Pilot (Planned) ................................................................................................................. 18 2.6 Public Wi-Fi ............................................................................................................................................. 19 2.7 Broadband Service to Residents and Businesses ........................................................................ 19 3 A Detailed Needs Assessment Identified Requirements, Concerns, and Priorities Related to Fiber Expansion ............................................................................................................................................. 21 4 Use Cases for Fiber and Wireless Networking in Cupertino .......................................................... 23 4.1 Use Case 1: Expand Fiber and Wi-Fi Network for City and Public-Facing Services ........... 23 4.2 Use Case 2: Expand Fiber to Create Resiliency and Enable Wi-Fi for City and Public- Facing Services in Emergency Situations .................................................................................................... 24 4.3 Use Case 3: Expand Fiber for City Use and Lease Excess Fiber to Businesses and Wireless Providers ............................................................................................................................................................... 25 5 Options and Costs for Expanding the City’s Fiber Network to Meet Current and Future Needs 27 5.1 Constructing New City-Owned Fiber to Serve All Proposed Use Cases Would Cost an Estimated $4.8 to $9.2 Million .......................................................................................................................... 28 5.1.1 System-Level Design and Construction Cost Estimates ....................................................................... 28 City of Cupertino | Fiber Optic Master Plan | August 2020 iii 5.1.2 Assumptions Underpinning Fiber Cost Estimates ................................................................................ 36 5.1.3 Network Maintenance Requirements ................................................................................................. 40 5.2 Alternative Approach: Leveraging Existing-Third Party Provider’s Fiber to Meet Current and Future City Needs ..................................................................................................................................... 45 5.2.1 Dark Fiber IRU Construction and Lease Costs ...................................................................................... 45 5.2.2 Dark Fiber Would Deliver Less Functionality Than City-Owned Fiber ................................................. 47 5.3 Summary of Estimated Fiber Expansion Costs ............................................................................. 48 5.4 Estimated Network Electronics Costs Would Be the Same for City-Owned Fiber and Dark Fiber ............................................................................................................................................................. 50 5.4.1 Electronics to Operate Fiber to City Facilities ...................................................................................... 50 5.4.2 Electronics to Operate Fiber to Traffic Intersection ............................................................................ 50 5.4.3 Electronics to Operate Fiber to City Parks and Downtown Wi-Fi ........................................................ 51 5.4.4 Electronics to Operate Fiber to CCC Emergency Sites ......................................................................... 53 5.4.5 Electronics to Operate Fiber Providing City Network Redundancy ..................................................... 55 6 Potential Fiber Network Business Plans ............................................................................................. 56 6.1 City-Owned and Operated Fiber Network .................................................................................. 56 6.2 Full Commercial Approach .............................................................................................................. 57 6.3 Hybrid Approach ................................................................................................................................. 58 7 Dig Once Policy Recommendation .................................................................................................. 59 7.1 Criteria .................................................................................................................................................... 59 7.2 Coordination with Excavators .......................................................................................................... 60 7.3 Overview Specifications .................................................................................................................... 60 7.4 Process .................................................................................................................................................... 66 7.5 Bidding Consideration ....................................................................................................................... 69 7.6 Construction Requirements .............................................................................................................. 69 7.7 Leadership and Process Considerations ....................................................................................... 69 Appendix A: City Fiber Network Applications ......................................................................................... 70 Appendix B: Needs Assessment Interviews .............................................................................................. 71 Appendix C: Fiber Lease Checklist............................................................................................................ 72 Appendix D: Proposed Future Routes ........................................................................................................ 73 Appendix E: Fiber Count and Ownership ................................................................................................. 74 City of Cupertino | Fiber Optic Master Plan | August 2020 iv Figures Figure 1: Existing City-Owned Fiber ........................................................................................................ 4 Figure 2: Use Case 1 – Expanded Fiber and Wi-Fi Network for City and Public-Facing Services ..... 6 Figure 3: Existing City-Owned Fiber ...................................................................................................... 12 Figure 4: Congestion in a City Network Handhole .............................................................................. 15 Figure 5: Fiber Termination in Traffic Signal Cabinet for Signal Control ............................................. 16 Figure 6: AT&T’s Fiber-to-the-Premises Service Availability in the City .............................................. 20 Figure 7: Use Case 1 – Expanded Fiber and Wi-Fi Network for City and Public-Facing Services ... 23 Figure 8: Use Case 2 – Expand Fiber to Create Resilient Fiber and Wi-Fi Network for City and Public-Facing Services in Emergency Situations ........................................................................ 25 Figure 9: Use Case 3 – Expand Fiber for City Use and Lease Excess Fiber ........................................ 26 Figure 10: Existing City-Owned Fiber .................................................................................................... 30 Figure 11: Fiber Construction to Expand Connectivity to City Facilities ............................................ 31 Figure 12: Fiber Construction to Expand Connectivity to Intersections ............................................. 32 Figure 13: Fiber Construction to Expand Connectivity to Parks and the Downtown Area ............... 33 Figure 14: Fiber Construction to Expand Network Redundancy ........................................................ 34 Figure 15: Fiber Construction to Connect CCC Emergency Sites ...................................................... 35 Figure 16:Smart Pole Depiction ............................................................................................................. 52 Figure 17: CCC Off-Grid Solar-Powered Smart Pole ........................................................................... 54 Figure 18: Typical Diagram – Major Corridor – Dig Once Coordination with a Communications Excavator ....................................................................................................................................... 62 Figure 19: Typical Diagram – Major Corridor – Dig Once Coordination with a Water, Power, or Sewer Excavator ............................................................................................................................ 63 Figure 20: Typical Diagram – Dig Once Coordination with a Communications Excavator ............ 64 Figure 21: Typical Diagram – Dig Once Coordination with a Water, Power, or Sewer Excavator .. 65 Figure 22: High-Level Dig Once Process Flowchart – Project Review ................................................ 67 Figure 23: High-Level Dig Once Process Flowchart – Implementation .............................................. 68 City of Cupertino | Fiber Optic Master Plan | August 2020 v Tables Table 1: Construction Cost Estimates by Phase ..................................................................................... 7 Table 2: Construction Cost Estimates by Phase ................................................................................... 29 Table 3: Construction Cost Estimates for Fiber to City Facilities ......................................................... 31 Table 4: Construction Cost Estimates for Fiber to Traffic Intersections ............................................... 32 Table 5: Construction Cost Estimates for Fiber to City Parks and the Downtown Area .................... 33 Table 6: Construction Cost Estimate for Fiber Segments to Increase Redundancy ......................... 34 Table 7: Construction Cost Estimates for Fiber to CCC Emergency Sites .......................................... 35 Table 8: Estimated Fiber Maintenance Costs ....................................................................................... 41 Table 9: Estimated Dark Fiber IRU Pricing for Fiber Lease ................................................................... 46 Table 10: Estimated Dark Fiber Mileage Required .............................................................................. 46 Table 11: Total Estimated Fiber Expansion Costs ................................................................................. 49 Table 12: Estimated Network Electronics Costs for New Fiber Segments .......................................... 50 Table 13: Estimated Network Electronic Costs for Fiber to City Facilities .......................................... 50 Table 14: Estimated Network Electronic Costs for Fiber to Traffic Intersections ................................ 51 Table 15: Estimated Network Electronics Costs for Fiber to City Parks and Downtown Wi-Fi .......... 51 Table 16: Estimated Costs for Adding Smart Poles at City Parks and in Downtown Area ................ 53 Table 17: CCC Smart Pole Cost Estimates ............................................................................................ 55 Table 18: Estimated Network Electronics Costs for Fiber Segments to Increase Redundancy ....... 55 Table 19: Technical and Business Activities for City Fiber Lease ....................................................... 72 City of Cupertino | Fiber Optic Master Plan | August 2020 1 1 Executive Summary The City of Cupertino requested that CTC Technology & Energy prepare this fiber optic master plan (and a companion amendment to the City’s wireless master plan) in late 2019. The City intended the study to include an assessment of its existing infrastructure, an identification of its needs for fiber infrastructure, a description of potential fiber use cases, and a high-level design and cost estimate for fiber construction (or dark fiber leases) to meet the City’s identified needs. Beyond these technical tasks, the City also sought to understand potential business models for constructing and operating additional fiber; whether any state or federal funding opportunities would be available to the City; and whether a Dig Once policy would enable cost-effective expansion of the fiber in the future. The City has successfully owned and operated its fiber network for more than 10 years— and the network has created real value for the City. Specifically, the City has offset the cost of leased circuits in connecting buildings and traffic infrastructure. The network has also provided more capacity, at a higher level of reliability, than would commercial services. Based on our review, the City’s fiber appears to be in good condition, with many years of useful life and the ability to operate, without modification, at thousands of times the speeds of current electronics. The City has full visibility into all portions of the fiber network, can scale to higher speeds simply by upgrading equipment, and has flexibility and spare capacity to add new locations. Having a dedicated fiber network provides many options for the City to address issues raised in the 2017 Cupertino Communications Risk Report prepared by the Cupertino Office of Emergency Services; concerns about communications resiliency for the City government and the public were also highlighted by the demonstrated vulnerability of commercial communications networks in the recent public safety power shutoff (PSPS) events. (These issues are discussed in the use cases presented in this report.) The Covid-19 quarantine underscores the criticality of broadband. While the City reports that broadband is available to all residents and that almost all residents can afford it, the City can use its fiber network, in combination with backup power and enhancement of City of Cupertino | Fiber Optic Master Plan | August 2020 2 Wi-Fi, to create an additional safety net of broadband for those who lose their service in outages or cannot afford the service. The fiber network also provides the underpinning of future Smart City infrastructure and services under consideration by the City. While most of these services require wireless connectivity, fiber will in many cases be necessary to connect wireless components, as well as to connect to cloud and back-office components. Early on, the City used best practices in funding and obtaining value from the network: 1. The City leveraged grant funding—almost all construction cost was covered by federal and State funds 2. The City partnered with the Silicon Valley Intelligent Transportation System (SV- ITS) initiative and Santa Clara County, which ensured the network construction created value for a wide range of stakeholders 3. The City recognized it could use the fiber both for transportation communications and other City government needs However, there are areas where practices could be improved to create even more value and efficiency. First, the network would be more valuable and reliable if the City had a contract in place to repair damage to outside cable plant in a committed timeframe. Having a contract would increase the City’s ability to rely on applications that are in the cloud or centralized at City Hall or the City’s disaster recovery facility—and would increase the reliability of network connections between buildings and the outside. The increased reliability could also be valuable to potential external stakeholders (such as wireless service providers or large businesses or institutions) that the City may wish to connect. Second, the City’s network has some (but not an overwhelming amount of) excess fiber capacity that the City may wish to consider leasing or trading. Some city and state governments have bartered strands from their excess capacity and significantly expanded their footprints. One possibility would be to trade excess fiber strands on arterial routes (where most City fiber is located) for fibers to sites in neighborhoods near parks and schools (where Crown Castle has previously constructed fiber). At a high level, this report recommends the City consider the following steps: City of Cupertino | Fiber Optic Master Plan | August 2020 3 1. Construct new City-owned and operated fiber to connect high-priority City facilities, increase the resiliency of the City’s existing fiber network, enable emergency communications, and expand the availability of Wi-Fi in parks and the downtown area; 2. Distribute a request for information (RFI) to dark fiber carriers to evaluate the feasibility and costs of leasing existing dark fiber from commercial providers as an alternative to constructing some or all of the recommended new fiber; and 3. Distribute a request for proposals (RFP) to internet service providers to procure high-quality, cost-effective managed services to connect low-criticality sites—to ensure that the City has selected the best available commodity network service for those locations. 1.1 Overview of Existing Infrastructure The City operates a 12.7-mile fiber network (Figure 1) that was built leveraging State and federal funds in a joint project with Santa Clara County as part of the Silicon Valley Intelligent Transportation System (SV-ITS) project in 2008. City of Cupertino | Fiber Optic Master Plan | August 2020 4 Figure 1: Existing City-Owned Fiber The network interconnects many of the City’s government buildings and most of its traffic system and can operate using generators at the buildings during extended power outages. (See Section 2 for an overview of current infrastructure.) 1.2 Needs Assessment A needs assessment process conducted for this study (Section 3) identified a number of City requirements for expanding its existing fiber network: 1. Providing fiber connectivity to five City facilities to replace leased services: Blackberry Farms Golf Course, Blackberry Farms, McClellan Ranch, Monta Vista Recreation Center, and the Human Resources Department. City of Cupertino | Fiber Optic Master Plan | August 2020 5 2. Constructing redundant fiber paths to the two core sites of the City network: City Hall and the Service Center. These redundant paths will reduce the risk of outages of the City’s network. 3. Providing fiber connectivity to the City’s 14 major traffic intersections that currently are not connected to the fiber network. These include intersections along North Wolfe Road, Perimeter Road and Vallco Parkway, Miller Avenue and Calle De Barcelona, Miller Avenue and Phil Lane, Rainbow Drive and Stelling Road, Bubb Road and McClellan Road, Bubb Road and Results Way, Stelling Drive and Greenleaf Drive, Homestead Road and Heron Avenue, and Foothill Boulevard and Voss Avenue. Such connectivity will support traffic operations and enable future Smart City applications. 4. Providing fiber connectivity to 13 of the City’s parks and the downtown area so the City can potentially provide free Wi-Fi to residents as well as other network capabilities. 5. Providing fiber connectivity to emergency sites used by the Cupertino Citizen Corps (CCC), which can be used by emergency personnel as well as to enable free public Wi-Fi services for emergencies and for City residents who have no or very limited access to broadband. These locations include the ARKs, fire stations, the Senior Center, and other sites used by CARES, CERT, and/or MRC. (See Section 2.4.) 6. Constructing new fiber segments that will increase the redundancy in the City’s fiber network. 7. Providing redundant solar backup power and other infrastructure hardening to improve the function and resiliency of the fiber network and critical end-points during wide-spread emergencies, including prolonged power outages. Redundant fiber adds value because it would improve the network’s resiliency and ability to operate following a natural emergency, an accidental fiber cut, or other damage to one segment of the network. Depending on the configuration, redundancy would also enable the network to continue operating if City Hall loses its connection. City of Cupertino | Fiber Optic Master Plan | August 2020 6 1.3 Use Cases We developed fiber use cases to address the City’s needs (Section 4). These include a series of incremental approaches to constructing fiber. The use cases are: 1. Construction of fiber to connect additional buildings and traffic intersections and expand public-facing Wi-Fi (Figure 2) 2. Enhancing the resiliency of the network to support continued operations in a long- term power outage or other emergency and provide Wi-Fi to the public during that emergency 3. Laying the groundwork for lease of excess City fiber to wireless providers or large institutional or business users. Figure 2: Use Case 1 – Expanded Fiber and Wi-Fi Network for City and Public-Facing Services 1.4 Cost Estimates for Expanding the City’s Fiber (Construction or Dark Fiber Lease) To examine the feasibility of the City constructing new fiber to expand its existing network, we developed a high-level design and cost estimate for network extensions aligned with the City’s needs. We estimate the City would incur capital costs of roughly $4.8 million to $9.2 million1 if it were to expand its fiber to connect additional buildings 1 Fiber cost estimates are based on costs seen by SV-ITS on the low end and the City’s cost for constructing fiber on the high end. The City will improve its chances of reducing costs by planning large- scale construction projects (which create economies of scale) and by coordinating fiber construction with capital projects such as road improvements and construction by non-City excavators (see Section 7). City of Cupertino | Fiber Optic Master Plan | August 2020 7 and traffic signals, enable public Wi-Fi, add redundancy, and connect the CCC emergency sites. The following table outlines the costs for each phase as well as a total cost of construction. The City could elect to build the second use case without constructing fiber to every City park in order to reduce the overall cost of the fiber expansion. The phases and further detail are provided in Section 5.1. Table 1: Construction Cost Estimates by Phase Phase Use Case Miles of New Construction Low Estimate High Estimate Connect City Facilities/ Create Redundancy to Core Sites 1 3.3 $900,000 $1.7 million Connect Major Traffic Intersections 1 3.3 $880,000 $1.8 million Connect City Parks and Downtown Wi-Fi 1 6.2 $1.7 million $3.1 million Use Case 1 Subtotal 12.8 $3.5 million $6.6 million Add Redundancy to the City’s Fiber Network 2 2.4 $640,000 $1.3 million Connect CCC Emergency Sites 2 2.5 $700,000 $1.3 million Use Case 2 Subtotal 4.9 $1.3 million $2.6 million Total 17.7 $4.8 million $9.2 million As an alternative to constructing its own fiber, we examined a scenario in which the City leases fiber from existing providers. We recommend the City investigate this option further, by way of a request for information (RFI) to fiber providers in the City (including Crown Castle, AT&T, Verizon, and Zayo). However, our analysis indicates that this approach may require almost as much fiber construction by the providers as a City- construction approach—and hence provide few savings. However, a leased fiber approach may be of value if used strategically, such as to connect buildings and traffic signals near a provider’s existing fiber. In terms of ongoing operations for both its existing fiber and any new construction, we recommend the City identify a suitable on-call maintenance contract to minimize the duration of outages. Currently the City bids maintenance work on an as-needed basis— which may lead to days or weeks of outages. We estimate the contractor cost for maintenance and required location of underground cables on the existing fiber network to be approximately $7,000 per month. City of Cupertino | Fiber Optic Master Plan | August 2020 8 Similarly, the City might be able to address reported reliability problems with its Comcast-provided network services by initiating a competitive bidding process—which could lead either to Comcast improving its service or to the City leasing a presumably better service (i.e., more capable, more reliable) from another provider. As part of the procurement, we recommend the City require bidders to demonstrate their proposed routing, network design, backup power, and guaranteed service levels. (As an illustration of the type of service the City seeks, it could refer to the high-availability service that Comcast proposes to deliver to support “vital community service” in a recent response to the California Public Utilities Commission’s inquiry into public safety power shutoff events.)2 1.5 Business Models and Funding Options for Fiber Expansion Section 6 comprises an evaluation of potential business models through which the City could expand its fiber network. We considered potential business models identified by the City: 1. The City owns and operates the fiber network 2. The City outsources operation and management of the network (with City or third-party ownership)—referred to as a “commercial approach” 3. A hybrid approach We recommend a hybrid approach, in which the City maintains ownership of the fiber, yet contracts with an on-call repair company to perform repairs within a specified time, and also strategically considers leasing fiber strands instead of building fiber to new locations where a fiber provider can offer a competitive total cost of operations. The City has attained a high level of performance using a relatively low amount of staffing and continues to obtain value in the existing arrangement with many more useful years of life in the fiber. The City could explore ways to derive revenue from its excess fiber and continue to operate a reliable and cost-effective service, leveraging the investments already made. Some cities that have excess fiber in their city-owned 2 Comcast Letter to CPUC, November 18, 2019, https://www.cpuc.ca.gov/uploadedFiles/CPUCWebsite/Content/News_Room/NewsUpdates/2019/Nov.% 2018%202019%20Comcast%20Response%20to%20President%20Batjer%20Nov.%2013%20Letter.pdf (accessed May 5, 2020). City of Cupertino | Fiber Optic Master Plan | August 2020 9 networks have explored opportunities. These include the City of Holly Springs, North Carolina, which leases fiber to Ting Internet (a provider of broadband fiber-to-the-home services) and Braintree (Massachusetts) Electric Light Department, which leases fiber to wireless providers. If the City chooses to lease or trade fiber (e.g., to wireless service providers that want to connect antenna sites, and to enterprise customers that want to operate their own connections between facilities), it would need to analyze the amount of fiber strands available on a given route and determine how many strands it would reasonably consider to be excess to its long-term needs. The City’s excess fiber count is limited. Depending on the route, only one-half to one- third of the City’s fiber is available—approximately 10 to 16 strands. Taken together, these strands may only connect a handful of additional locations, thus limiting the fiber’s appeal and also the price that a third party would pay. And on many routes, especially on Stevens Creek Blvd., many or all fiber strands are limited to SV-ITS traffic management use. However, the excess fiber may provide significant value to the City for connecting new locations and enhancing the value of the network to the City. 1.6 Recommendation for a Dig Once Policy Looking to the future, the City might address the high cost of fiber construction to add new sites by instituting a Dig Once requirement that would capitalize on fiber builds by wireless providers or other excavators (and would aim to reduce pavement cuts and preserve the limited area within the public right-of-way). For example, the City could request in-kind contributions of fiber in agreements with wireless providers or in exchange for construction in the right-of-way; the City might also pay only the incremental cost for adding fiber strands during other entities’ fiber construction. Section 7 provides Dig Once recommendations and draft typical designs for configuration of City conduit in a range of Dig Once scenarios. City of Cupertino | Fiber Optic Master Plan | August 2020 10 2 Overview of Existing City Communications Infrastructure and Related Assets CTC reviewed GIS data and AutoCAD drawings provided by City personnel and gathered input and insight during the needs assessment process (see Section 3). This section describes the City’s fiber network (including physical infrastructure, operations, and applications), traffic communications infrastructure, emergency operations center, CCC emergency intranet, and public-facing Wi-Fi, as well as the retail broadband services available to residents. 2.1 Data Related to City’s Infrastructure The City has an extensive and comprehensive GIS database of assets; some of which is available to the public through the City’s website.3 The data provided by the City for this study included: • Infrastructure • Buildings and addresses • Roads and trails • Land and boundaries The infrastructure data layers include the assets that the City uses to run its transportation and IT networks. The assets in the infrastructure group include transportation equipment such as traffic cabinets, cameras, radar signs, and signal heads. The infrastructure data also contains the City’s fiber optic routing/conduit, splice locations, and pull box locations, as well as the locations of connected facilities. The fiber data contains information such as the fiber’s owner, how many strands are within the fiber, and who maintains the fiber. The infrastructure data also has information on wireless siting within the City and potential small cell sites such as streetlights. The GIS data has the existing macro siting locations, which may be on government property, as well as the locations where small cells have been deployed. The City streetlight data provides the location of City-owned streetlights. 3 “GIS Open Data Portal,” City of Cupertino, https://www.cupertino.org/businesses/business- resources/gis-open-data-portal (accessed May 6, 2020). City of Cupertino | Fiber Optic Master Plan | August 2020 11 City-owned streetlights are in the underground utility areas, comprising about half of the City, whereas Pacific Gas & Electric (PG&E) owns the streetlights outside of the underground areas. Streetlights are a potential location for future small cell wireless siting. The streetlight metadata provides information on who owns the streetlight, who maintains the streetlight, the pole type, and the pole height. These data could be useful for wireless carriers looking for locations to site their facilities. The building and addresses group contains address points for each property, including addresses for multi-dwelling units (MDU) and large office buildings. The data also include building footprint information such as the building height and the number of stories. The building and address data may be useful for wireless carriers to determine where they can locate new facilities and how many addresses and people their wireless facilities will serve. The roads and trails group contains the street centerlines, bike path, and trail layers. These layers help identify routes for future fiber routing. The land and boundaries group contains the City’s boundaries, parcels, easements, and government properties. These layers can be helpful for identifying the owners of property where a wireless carrier may want to site new facilities or to determine where future fiber may be needed. The City also maintains CAD data on the details of its fiber deployment such as splicing data and fiber strand assignments. 2.2 Fiber Optic Network The City’s fiber network consists of 12.7 miles of fiber that primarily was constructed underground in conduit (Figure 3, with a more detailed map in Appendix E). The fiber ranges from 12 to 144 count, with most main routes having 96 or 144 count. While this count is substantially less than the count in many current backbone fiber routes (often 432 or more), it is typical of the fiber count installed for municipal and traffic communications purposes. City of Cupertino | Fiber Optic Master Plan | August 2020 12 Figure 3: Existing City-Owned Fiber Construction Background and Restrictions on Use The majority of the City’s fiber was built jointly with Santa Clara County as part of the Silicon Valley Intelligent Transportation System (SV-ITS) project in 2008. The $5.2 million cost was covered mostly through Federal Highway Administration (FHWA) Congestion Mitigation and Air Quality (CMAQ) funds.4 The parties to the agreement are the City, Santa Clara County, the City of San Jose, and Caltrans. Different portions of the network have different apportionment of fiber strands between the County and the City. Agreements provide for use of fiber by the City in exchange for its installation in the City’s right-of-way; the agreements limit the use of the SV-ITS assigned fiber to the regional transportation management system installed by the SV-ITS. 4 “Congestion Mitigation and Air Quality Improvement (CMAQ) Program,” Federal Highway Administration, U.S. Department of Transportation, https://www.fhwa.dot.gov/environment/air_quality/cmaq/ (accessed April 2020). City of Cupertino | Fiber Optic Master Plan | August 2020 13 The assignment of fiber to SV-ITS creates most limitations on the routes on Stevens Creek Blvd., were most of the strands west of De Anza Blvd. and all of the strands east of De Anza Blvd. are assigned to SV-ITS (Appendix E). Thus, while U.S. Department of Transportation policies have been changing in recent years to allow more flexibility and encourage a broader range of uses of fiber, the City would need to secure the agreement of the parties to the SV-ITS agreements to use those strands for other uses. (See Section 6 for details on potential business models, including the leasing of excess fiber.) For example, in order to add the parks in Section 3, strictly speaking it may be necessary to seek approval from SV-ITS parties. Routes and Connections The City’s enterprise fiber network connects seven sites with a 1 Gbps connection. The key sites are City Hall, which is the center-point of the network, and Quinlan Community Center, which is an alternate connection site to the City’s disaster recovery center. Sites on the City’s enterprise fiber network are: • City Hall • Community Hall • Service Center • Senior Center • Sports & Teen Center • Quinlan Community Center • Traffic Operations Center Five City government sites are connected via a Comcast Business Cable Internet (i.e., managed service) at 100 Mbps/20 Mbps (upload/download). These sites are City Hall and the Service Center (which have primary connections on fiber and secondary connections on VPN) and five others which are not connected to the City’s fiber network because they are lower-criticality sites. The five sites on VPN are: • McClellan Ranch • Blackberry Farm Retreat • Blackberry Farm Golf Course • Monta Vista Recreation Center • Human Resources Office City of Cupertino | Fiber Optic Master Plan | August 2020 14 Because they have less robust connections—and because the City’s Comcast service experiences periodic outages—the City is exploring other options for the service. At Blackberry Farm and McClellan Ranch, the City identified fiber connections to these sites as a future requirement (see Section 3). Operations The network’s operational target is 99.9 percent uptime during normal operations, with up to eight hours restoration time in an emergency. The City’s bandwidth requirements are expected to increase. For the near-term future, it will need to deliver 100 Mbps to any facility. For future fiber expansions, it will need to deliver 1 Gbps and be 10 Gbps-ready. The City’s desired latency over the network is less than 100 ms with a jitter of 30 ms or better for facilities within City limits. The City purchases internet service from AT&T; that link is located at City Hall. Additionally, The City purchases redundant internet service from Zayo, which built fiber from its network to meet the City’s fiber on Stelling Road, near Stevens Creek Blvd. The City’s network connects to its disaster recovery (DR) center in Arizona via 200 Mbps connections over links capable of 1 Gbps.5 The DR supports backup of applications hosted at City Hall, including the City’s enterprise resource planning (ERP), access control, and file server. The City’s IT and communications services are not funded by chargebacks to departments. Rather, the City prepares a three-year strategic plan and a one-year budget. An October 2017 strategic plan is being updated; if a needed initiative is not in the plan, the City seeks to put it in the current or upcoming fiscal year budget. The City reports that its IT staff level currently is adequate for IT and network needs. Applications The City’s high-bandwidth applications include video conferencing by Zoom and additional conferencing features provided by Office 365. For VoIP, the City uses a Mitel ShoreTel VoIP system that is located at City Hall with disaster recovery/high availability service at the Service Center. 5 The current usage can be expanded upon request to 1 Gbps if the City’s demand increases. City of Cupertino | Fiber Optic Master Plan | August 2020 15 The City is implementing IP-video cameras. The first phase will be in “areas that have a monetary interaction with members of the public.” Phase two will include common areas. The City is increasingly using cloud services (Microsoft 365) to store City files. The City’s goal is to be completely cloud-based for other systems, including ERP, Accela (land management application), and ActiveNet (parks and recreation system). The City’s staff use laptop computers so they can work from wherever they are. City network access is provided by a secure VPN. The City’s applications are listed in Appendix A. Conduit and Handholes The City’s conduit is installed at varying depths (much of it not at the standard 2-foot depth) and has varying levels of congestion. Because some congestion exists in the City’s handholes (see an example in Figure 4), expansion of the City’s fiber network may require the installation of larger handholes. Figure 4: Congestion in a City Network Handhole City of Cupertino | Fiber Optic Master Plan | August 2020 16 Traffic Communications The City’s traffic communications center is located at the City’s Traffic Operations Center. Traffic communications operations use a mixture of Ethernet and serial communications to fiber-connected cameras and signals (Figure 5); the department is interested in improving reliability and scalability by upgrading to full Ethernet to each fiber-connected signal. Figure 5: Fiber Termination in Traffic Signal Cabinet for Signal Control City of Cupertino | Fiber Optic Master Plan | August 2020 17 The department also wants to connect signals that currently do not have fiber links, including a string of intersections along Wolfe Road that currently are served by twisted- pair copper cable: 1. Wolfe Road/Vallco Parkway 2. Wolfe Road/280 S. Ramp 3. Wolfe Road/280 N. Ramp 4. Wolfe Rd/Pruneridge Avenue 5. Wolfe Road/Apple Parkway Entry Other signals that have no connection at all are located at these intersections: 1. Miller Avenue/Calle De Barcelona 2. Miller Avenue/Phil Lane 3. Rainbow Drive/Stelling Road 4. Bubb Road/McClellan Road 5. Bubb Road/Results Way 6. Foothill Boulevard/Voss Avenue 7. Stelling Road/Greenleaf Drive 8. Homestead Road/Heron Avenue One more signal, at Perimeter Road and Vallco Parkway, might have a conduit running to Wolfe Road, but it is not connected. 2.3 Emergency Operations Center Police, fire, emergency medical services, and dispatching are operated by Santa Clara County, so those functions are not included in this analysis. The City’s emergency operations center (EOC) is located upstairs in City Hall. Because the City understands that City Hall’s clay roof may collapse in an earthquake, the City will need to be able to operate the EOC function in an alternate facility. The City’s emergency command vehicle can support this role as well. City of Cupertino | Fiber Optic Master Plan | August 2020 18 2.4 ARKnet ARKnet6 is Cupertino’s wireless emergency intranet. Overseen by volunteers—including the Cupertino Citizens Corps (CCC) and Cupertino Amateur Radio Emergency Service (CARES)—ARKnet is an emergency wireless network that connects critical CCC locations together (primarily the ARKs) and is designed for use by the CCC and other City staff during emergencies and for training purposes. ARKnet supports the City’s need for an accurate and timely formation of the situation in the field during an emergency activation. ARKnet supplements existing communication capabilities at the ARKs that enable residents, emergency response personnel, and other City staff to convene and communicate at recovery and shelter locations in the event of an emergency (e.g., an earthquake, dam failure, or public safety power shutoff (PSPS)), particularly if cellular networks or commercial wireline networks are down). The CARES and CERT teams are trained to provide emergency communications and logistics support to the various CCC sites around the City in the event of an emergency activation. ARKnet supplements the existing voice and packet communications from CARES and utilizes 5.8 GHz unlicensed wireless radio equipment to allow for video, phone, and file sharing services to the various sites on the network. All sites are either completely off- grid and solar-powered or are co-located with emergency backup power (e.g. at City Hall and the Service Center). A central site, located at Lehigh Permanente Cement, serves as the primary backbone for this network. ARKnet is designed as an isolated emergency network, not dependent on the City’s infrastructure or grid power. The primary way that this plan can support ARKnet is in providing provisions for installation of radio equipment (including masts, equipment, and/or solar panels) into new and existing construction three stories tall or higher, specifically those at elevated locations or heights. 2.5 Smart City Pilot (Planned) The City does not currently operate Smart City infrastructure, but plans to launch an air quality sensors pilot in 2021 with about 20 locations (likely connected wirelessly). The City is also considering traffic applications—including 24/7 counts, video analytics, and 6 Marcel Stieber, “ARKnet Wireless Emergency Intranet,” Presentation (Pacificon Amateur Radio Convention), Oct. 15, 2016. See also: “ARKnet Deployment System Design Document,” Cupertino Citizens Corps, January 2016, https://www.cupertinoares.org/pdocs/ARKnet-Deploy-SDD.pdf (accessed April 30, 2020). City of Cupertino | Fiber Optic Master Plan | August 2020 19 adaptive traffic signaling (in support of efforts to eliminate traffic fatalities and injuries, also known as Vision Zero). Based on a cost-benefit analysis in 2010, when LED streetlighting was a new technology, the City chose not to install LED lighting but opted instead for induction lamps. Induction lighting has very similar power costs as LED (induction lamps are slightly higher), but lacks many of the Smart City features currently available for streetlights. 2.6 Public Wi-Fi Public Wi-Fi is available at all City facilities. The City has interest in deploying Wi-Fi as a free public amenity in other locations (see Section 3). 2.7 Broadband Service to Residents and Businesses The City has indicated that the “digital divide” is not an issue and thus is not the City’s policy priority. Most of the City has Comcast cable modem service. According to the City, AT&T has built fiber-to-the-premises in some but not all of the City—so about half of the City has competition between two high-speed services. The map in Figure 6, which is based on data self-reported by carriers on the Federal Communications Commission’s Form 477, shows that AT&T’s fiber is mostly in the eastern portion of the City. City of Cupertino | Fiber Optic Master Plan | August 2020 20 Figure 6: AT&T’s Fiber-to-the-Premises Service Availability in the City Residents report less reliability of wireline and wireless broadband services on both AT&T and Comcast’s networks west of CA-85. The City reports spotty wireless services on Verizon and T-Mobile’s networks during a public safety power shutoff (PSPS) in October 2019, when 30 sites (2 percent) in Santa Clara County and 134 sites (50 percent) in Marin County were out of service.7 While the City’s internet continued to function during the PSPS, the City has an interest in expanding current satellite internet as a backup for future emergencies. We recommend the City focus on currently available satellite options but continue to evaluate low-earth orbit options such as Starlink as they become available, in order to obtain higher speeds and lower latency. 7 Communications Status Report for Areas Impacted by California Public Safety Power Shutoffs, October 27, 2019, https://docs.fcc.gov/public/attachments/DOC-360454A1.pdf, accessed April 15, 2020. City of Cupertino | Fiber Optic Master Plan | August 2020 21 3 A Detailed Needs Assessment Identified Requirements, Concerns, and Priorities Related to Fiber Expansion CTC met with City staff to discuss project goals and objectives, review relevant maps and documents, and address the team’s primary concerns—including issues raised in the Cupertino Communications Risk Report.8 We then held in-depth discussions with representatives of City departments to identify specific needs that will be addressed by this plan and a complementary amendment to the City’s wireless master plan. (A list of discussion participants is included in Appendix B.) We identified the City’s following key fiber needs, which are addressed in the use cases and the candidate fiber network designs below: 1. Providing fiber connectivity to five City facilities to replace leased services: Blackberry Farms Golf Course, Blackberry Farms, McClellan Ranch, Monta Vista Recreation Center, and the Human Resources Department. 2. Constructing redundant fiber paths to the City network’s two core sites: City Hall and the Service Center. These redundant paths will reduce the risk of outages on the City’s network by ensuring that a single fiber break or loss of a single site will not cut off the City’s fiber network. Even though these are short routes, redundant paths out of City Hall and the Service Center are critical for City IT operations. 3. Providing fiber connectivity to 14 major traffic intersections that currently are not connected to the fiber network. These include intersections along North Wolfe Road, Perimeter Road and Vallco Parkway, Miller Avenue and Calle De Barcelona, Miller Avenue and Phil Lane, Rainbow Drive at Stelling Road, Bubb Road and McClellan Road, Bubb Road and Results Way, Stelling Road and Greenleaf Drive, Homestead Road and Heron Avenue, and Foothill Boulevard and Voss Avenue. Such connectivity will support traffic operations and enable future Smart City applications. 4. Providing fiber connectivity at three locations in the downtown area and to 13 of the City’s parks, so the City can potentially provide free Wi-Fi to residents as a public amenity (i.e., not for the City’s internal needs) as well as other network 8 “Cupertino Communications Risk Report,” City of Cupertino Office of Emergency Services, February 7, 2017. City of Cupertino | Fiber Optic Master Plan | August 2020 22 capabilities. For our model, we estimate the cost of three separate access points in the downtown area—at the intersections of Stevens Creek and N. Tantau Avenue, Stevens Creek and N. Wolfe Avenue., and Vallco and Main. The City’s 13 identified park locations for Wi-Fi access are: • Creekside Park – 10455 Miller Avenue • Franco Park – 10981 Franco Court • Hoover Park – Leeds Avenue near Primrose • Jollyman Park – 1000 S. Stelling Road • Linda Vista Park – Linda Vista Drive near Columbus • Memorial Park – Stevens Creek and Mary Avenue (partial coverage currently provided via Quinlan Community Center and the Senior Center) • Monta Vista Park – Foothill Boulevard and Voss Avenue (partial coverage currently provided via the Cupertino Preschool and Monta Vista Recreation Center) • Portal Park – N Portal Avenue off Stevens Creek Boulevard • Somerset Square Park – Stokes Avenue near Peninsula Drive • Sterling Barnhart Park – 10486 Sterling Boulevard • Three Oaks Park – Candlelight Way near Rainbow Drive • Varian Park – Ainsworth Drive at Vista Knoll • Wilson Park – S Portal Avenue near Stevens Creek Boulevard 5. Providing fiber connectivity to sites used by the Cupertino Citizen Corps (CCC), which can be used by emergency personnel as well as to enable free public Wi-Fi services during emergencies. These locations include the ARKs, fire stations, the Senior Center, and other sites used by CARES, CERT, and/or MRC. 6. Constructing new fiber segments that will increase the redundancy in the City’s fiber network. We note that the City’s public schools and libraries are not included in this list because those sites are served in other ways. The Cupertino Library is a member of the Santa Clara County Library District and is connected to the County’s fiber network; the public school districts have fiber from commercial providers. A final identified need relates not to fiber expansion, but to a backup service. The City has satellite internet service at City Hall; there is interest in also having satellite as a backup service for the Service Center and the mobile command vehicle. City of Cupertino | Fiber Optic Master Plan | August 2020 23 4 Use Cases for Fiber and Wireless Networking in Cupertino The use cases described below address the City’s identified needs—and maximize the use and benefit of the City’s existing fiber. These examples—which build incrementally on each other—are informed by use cases in other jurisdictions nationwide. For example, municipal utilities in Massachusetts are providing dark fiber and pole attachments to wireless service providers. Similarly, the City of Baltimore, Maryland, provides access to city conduit and attachment on that city’s light poles for commercial wireless providers. Other U.S. cities, including New York, are also investigating the best ways to provide fiber in conjunction with colocation facilities and rooftop access to incent placement of competitive high-speed wireless services. 4.1 Use Case 1: Expand Fiber and Wi-Fi Network for City and Public-Facing Services In the most straightforward use case, the City would expand its fiber network to connect additional City facilities (both buildings and traffic infrastructure) and enable public- facing Wi-Fi on light poles and potentially from multi-function “smart poles” (Figure 7). This use case would leverage the City’s existing fiber to expand broadband access to the public and to enable future City monitoring and functionality. Figure 7: Use Case 1 – Expanded Fiber and Wi-Fi Network for City and Public-Facing Services Executing this use case would require the City to extend its existing fiber network to parks, recreation facilities, intersections, and public Wi-Fi locations. Public Wi-Fi in the downtown area and other high-traffic locations would be delivered via pole mounted hotspots or smart poles (which would have the functional and aesthetic advantage of enabling multiple communications functions in a single, compact footprint). City of Cupertino | Fiber Optic Master Plan | August 2020 24 We note that expanding fiber to the City’s parks would be relatively expensive (see Section 5.1.1) and may not provide the level of perceived value as some of the other stages; one way to limit these costs might be to use managed services to connect the parks. 4.2 Use Case 2: Expand Fiber to Create Resiliency and Enable Wi-Fi for City and Public-Facing Services in Emergency Situations Recognizing the City’s extensive planning for emergency response (see details on ARKnet in Section 2.4) and for the benefit of a higher bandwidth resilient communications infrastructure to citizens and emergency personnel at the CCC emergency locations, this use case focuses on expanding the City’s fiber to enable robust communications capabilities during emergency situations. This use case (Figure 8) addresses wired and wireless communications capabilities and is applicable across the range of emergency situations that might disrupt normal communications services; it addresses issues raised in the Cupertino Communications Risk Report.9 Like the first use case, this use case would support public-facing Wi-Fi and smart poles for day-to-day functionality in the downtown area and other high-traffic public locations. This use case would also require the City to create redundancy in the network (i.e., to ensure that City Hall is not the network’s single point of failure) and establish a backup internet connection (potentially via satellite) at City Hall and the Service Center. This use case would also leverage new, resilient fiber routes and power sources to enable the City to continue delivering broadband communications capabilities to City staff, Santa Clara County first-responders, and the public during extended power outages, quarantines, post-earthquake recovery periods, and other times of citywide disruption. Notably, this use case would build on the ARKnet concept in that it would include predetermined locations where residents can come to communicate and gain access to needed goods and services during an emergency situation. 9 “Cupertino Communications Risk Report,” City of Cupertino Office of Emergency Services, February 7, 2017. City of Cupertino | Fiber Optic Master Plan | August 2020 25 Figure 8: Use Case 2 – Expand Fiber to Create Resilient Fiber and Wi-Fi Network for City and Public-Facing Services in Emergency Situations Executing this use case would require the City to take a number of concrete steps to enhance the current fiber network (see Section 5.1.1). In addition to adding redundancy to eliminate a single point of failure at City Hall, the City would need to expand fiber routes in general to create rings for diversity. The City would need to install backup power at key locations on the network, including in-building sites, CCC locations, and in the parking lots of schools, shelter areas, and government buildings. At those outdoor locations, too, the City would need to install outdoor fiber and electronics for resilient Wi-Fi and CCC site connections. From an operational standpoint, we recommend the City secure an on-call contract for fiber repair to improve resiliency, taking care to obtain terms that guarantee response in a large-scale emergency.10 Similarly, the City would need to acquire a more resilient alternative to the Comcast cable modem service currently used at some locations. 4.3 Use Case 3: Expand Fiber for City Use and Lease Excess Fiber to Businesses and Wireless Providers Incorporating the construction and operational improvements described above, this use case would add a revenue-generating element (Figure 9). The City would lease excess fiber strands to wireless service providers and other potential users (such as large businesses). In addition to creating a revenue stream for the City (assuming it were successful in negotiating lease agreements), using the City’s fiber in this way might reduce the time needed for wireless service providers to deploy advanced services in the community—and could also create more competition in the dark fiber market, which would benefit enterprise customers. 10 For more details, see Section 5.1.33 and Section 6.1. City of Cupertino | Fiber Optic Master Plan | August 2020 26 Figure 9: Use Case 3 – Expand Fiber for City Use and Lease Excess Fiber Effectively executing this use case would require the City to conduct market outreach to determine likely needs and gaps among wireless providers. We recommend the City develop a pricing schedule and draft service-level agreement (SLA) in preparation for negotiations, as well as other steps described in Appendix C. City of Cupertino | Fiber Optic Master Plan | August 2020 27 5 Options and Costs for Expanding the City’s Fiber Network to Meet Current and Future Needs Expanding the City’s existing fiber optic network would address the City’s key needs in terms of reaching the City facilities and other locations not currently connected by City- owned fiber (in some cases, as an alternative to relying on commercial managed services) and creating redundancy and resiliency in the network. Expanding the City’s fiber may also offer long-term cost savings and provide technical advantages for the City—as well as enabling wireless service providers to expand their infrastructure in the City. In the sections below, we describe the two primary ways the City could consider expanding its existing fiber network to support the government-facing operations described in Section 4, as well as to serve as a resource for wireless providers and to enable public-facing Wi-Fi in public areas and future Smart City applications: 1. The City could construct and operate its own fiber 2. The City could lease third-party dark fiber (though, as we illustrate, constructing City-owned fiber is a better solution from both a cost and functional perspective) We recommend also that the City take steps to the service at City sites connected currently over Comcast cable modem service (which the City reports has not been as reliable as the City would like). We recommend the City seek a more robust and reliable managed service option through a competitive process in which the bidders (likely including Comcast as the incumbent provider) would be required to demonstrate their proposed routing, network design, backup power, and guaranteed service levels. As an illustration of the type of service the City seeks, it could refer to the high-availability service that Comcast claims to be capable of delivering to support “vital community service” in a response to the California Public Utilities Commission’s recent inquiry into PSPS events.11 11 Comcast Letter to CPUC, November 18, 2019, https://www.cpuc.ca.gov/uploadedFiles/CPUCWebsite/Content/News_Room/NewsUpdates/2019/Nov.% 2018%202019%20Comcast%20Response%20to%20President%20Batjer%20Nov.%2013%20Letter.pdf (accessed May 5, 2020). City of Cupertino | Fiber Optic Master Plan | August 2020 28 5.1 Constructing New City-Owned Fiber to Serve All Proposed Use Cases Would Cost an Estimated $4.8 to $9.2 Million In this section, we provide an overview of a technical approach and cost estimates developed to examine the feasibility of the City constructing new fiber to expand its existing network. The outside plant cost estimate ranges from $4.8 million to $9.2 million (itemized in Table 2 below)—including lateral construction, splicing, and fiber termination costs at each City facility that is connected—and assumes that the City maximizes the use of its existing fiber resources to reduce the overall cost of fiber deployment. Estimated network electronics costs are provided in Section 5.3. System-Level Design and Construction Cost Estimates CTC developed a system-level design for a fiber optic network to serve as the basis for estimating costs. Design priorities targeted by this conceptual design comprise the City’s identified needs (with separate costs identified for each priority): • Providing fiber connectivity to five City facilities to replace leased services: Blackberry Farms Golf Course, Blackberry Farms, McClellan Ranch, Monta Vista Recreation Center, and Human Resources. • Constructing redundant fiber paths to two core sites of the City network, City Hall and an alternate site at the Service Center. These redundant paths will eliminate the risk of a single failure causing an outage of the City’s network. • Providing fiber connectivity to the City’s 14 major traffic intersections that currently are not connected to the fiber network. These include intersections along North Wolfe Road, Perimeter Road and Vallco Parkway, Miller Avenue and Calle De Barcelona, Miller Avenue and Phil Lane, Rainbow Drive at Stelling Road, Bubb Road and McClellan Road, Bubb Road and Results Way, and Foothill Boulevard and Voss Avenue. • Providing fiber connectivity to 13 of the City’s parks and in the downtown area that will enable the City to provide free Wi-Fi to residents as well as other network capabilities. • Providing fiber connectivity to sites used by the Cupertino Citizen Corps (CCC), which can be used by emergency personnel as well as to enable free public Wi-Fi services during emergencies. These locations include the ARKs, Fire Stations, Senior Center, and other sites used by CARES, CERT, and/or MRC. City of Cupertino | Fiber Optic Master Plan | August 2020 29 • Constructing new fiber segments that will increase the redundancy in the City’s fiber network. We planned the new fiber routes in separate batches based on priority, with each phase incremental to the phase prior to it. The CCC sites are independent of the other phases in the event they need fiber connectivity faster than the other phasing. Fiber cost estimates are based on costs seen by SV-ITS on the low end and the City’s cost for constructing fiber on the high end.12 The following table outlines the costs for each phase as well as a total cost of construction. Table 2: Construction Cost Estimates by Phase Phase Use Case Miles of New Construction Low Estimate High Estimate Connect City Facilities/ Create Redundancy to Core Sites 1 3.3 $900,000 $1.7 million Connect Major Traffic Intersections 1 3.3 $880,000 $1.8 million Connect City Parks and Downtown Wi-Fi 1 6.2 $1.7 million $3.1 million Use Case 1 Subtotal 12.8 $3.5 million $6.6 million Add Redundancy to the City’s Fiber Network 2 2.4 $640,000 $1.3 million Connect CCC Emergency Sites 2 2.5 $700,000 $1.3 million Use Case 2 Subtotal 4.9 $1.3 million $2.6 million Total 17.7 $4.8 million $9.2 million The system-level design would expand the City’s extensive fiber optic network that connects City facilities and the City’s traffic system. The City’s existing fiber network consists of 12.7 miles of fiber that was constructed underground in conduit, mostly in coordination with the Silicon Valley ITS (SV-ITS) traffic communications fiber network. Figure 10 depicts the existing fiber network. 12 While the SV-ITS agreement for the City’s original fiber network construction referenced a cost of $50 per foot, City staff estimates that expanding the fiber network will cost approximately $100 per foot. This estimate is for mostly shorter-distance projects (e.g., one-quarter mile) and may not be representative of the cost for longer routes, where the mobilization cost can be spread among many miles of construction. City of Cupertino | Fiber Optic Master Plan | August 2020 30 Figure 10: Existing City-Owned Fiber The first priority would be to construct fiber to the five City facilities that are not connected to the fiber network and to provide redundant connections to City Hall and the Service Center to improve network resiliency and increase network availability. This City of Cupertino | Fiber Optic Master Plan | August 2020 31 would require approximately 3.3 miles of fiber construction at an estimated cost of $900,000 to $1.7 million. Figure 11 shows this first phase of fiber construction. Figure 11: Fiber Construction to Expand Connectivity to City Facilities The following table outlines the fiber construction costs per segment in this phase. Table 3: Construction Cost Estimates for Fiber to City Facilities Name Low Estimate High Estimate Service Center Redundancy $345,000 $627,000 City Hall Redundancy $99,000 $180,000 Human Resources $28,000 $51,000 Blackberry Farms Golf $11,000 $19,000 Blackberry Farm $135,000 $245,000 McClellan Ranch $180,000 $328,000 Monta Vista Recreation Center $114,000 $208,000 Total $900,000 $1.7 million City of Cupertino | Fiber Optic Master Plan | August 2020 32 The second phase of fiber construction would be to connect the 14 remaining major traffic intersections that are not connected to the fiber network. This would require approximately 3.3 miles of construction at an estimated cost of $880,000 to $1.8 million. Figure 12 shows the second phase of fiber connectivity; Table 4 outlines the costs for each segment. Figure 12: Fiber Construction to Expand Connectivity to Intersections Table 4: Construction Cost Estimates for Fiber to Traffic Intersections Name Low Estimate High Estimate North Wolfe Road $288,000 $576,000 Vallco Parkway and Perimeter Drive $40,000 $79,000 Miller Avenue and Calle De Barcelona $101,000 $203,000 Rainbow Drive and Stelling Road $145,000 $290,000 Bubb Road and Results Way $20,000 $39,000 Bubb Road and McClellan Road $115,000 $231,000 Miller Avenue and Phil Lane $23,000 $46,000 Foothill Boulevard and Voss Avenue $5,000 $10,00 Stelling Road Greenleaf Drive $69,000 $139,000 Homestead Road Heron Avenue $74,000 $148,000 Total $880,000 $1,761,000 City of Cupertino | Fiber Optic Master Plan | August 2020 33 The third phase of construction would be extending fiber to 13 of the City’s parks and the downtown area. This would require 6.2 miles of fiber construction at an estimated cost of $1.7 million to $3.1 million. Figure 13 shows the third phase of fiber connectivity. Figure 13: Fiber Construction to Expand Connectivity to Parks and the Downtown Area The following table outlines the costs for connecting each of the City’s 12 parks that do not already have fiber laterals. (The last park, Monta Vista Park, will have fiber from connecting the Monta Vista Recreation Center.) Table 5: Construction Cost Estimates for Fiber to City Parks and the Downtown Area Name Low Estimate High Estimate Franco Park $4,000 $7,000 Hoover Park $137,000 $248,000 Jollyman Park $109,000 $198,000 Linda Vista Park $186,000 $338,000 Memorial Park $33,000 $61,000 Creekside Park $162,000 $294,000 Portal Park $134,000 $244,000 Somerset Park $212,000 $386,000 Sterling Barnhart Park $309,000 $562,000 Three Oaks Park $164,000 $298,000 Varien Park $130,000 $236,000 Wilson Park $136,000 $248,000 Downtown Wi-Fi $15,000 $30,000 Total $1.7 million $3.1 million City of Cupertino | Fiber Optic Master Plan | August 2020 34 The fourth phase of construction (which, like the preceding phases, would be incremental to the phases before it) would be constructing fiber segments to increase the redundancy of the City’s fiber network (Figure 14, also shown in higher resolution in Appendix D). This would require 2.4 miles of fiber construction at an estimated cost of $640,000 to $1.3 million. If the City deems building fiber to the City parks too expensive, most of the redundancy links can be built without needing the City parks fiber laterals. Figure 14: Fiber Construction to Expand Network Redundancy The following table outlines the cost of the redundancy segments. Table 6: Construction Cost Estimate for Fiber Segments to Increase Redundancy Name Low Estimate High Estimate Redundancy Path 1 $60,000 $130,000 Redundancy Path 2 $260,000 $528,000 Redundancy Path 3 $180,000 $360,000 Redundancy Path 4 $130,000 $261,000 Total $640,000 $1.3 million City of Cupertino | Fiber Optic Master Plan | August 2020 35 The City could also construct fiber to the CCC emergency sites (excluding the Lehigh Permanente Quarry) independent of the other fiber expansion builds to provide fiber connectivity to personnel in an emergency and to support free Wi-Fi for residents during emergencies. This phase would require 2.5 miles of fiber construction (Figure 15) at an estimated cost of $700,000 to $1.3 million (Table 7). Figure 15: Fiber Construction to Connect CCC Emergency Sites Table 7: Construction Cost Estimates for Fiber to CCC Emergency Sites Name Low Estimate High Estimate Lawson Middle School $70,000 $128,000 De Anza College $12,000 $21,000 Creekside Park $162,000 $294,000 Garden Gate Elementary $126,000 $229,000 Regnart Elementary School $295,000 $537,000 Monta Vista Fire Station $35,000 $64,000 Total $700,000 $1.3 million City of Cupertino | Fiber Optic Master Plan | August 2020 36 Assumptions Underpinning Fiber Cost Estimates While not fully vetted in the manner necessary for permitting and construction, these fiber optic designs are likely to closely approximate a final design meeting the stated design objectives. The cost estimates in this report reflect a range of potential cost estimates. Additional outside plant engineering and field surveys would be needed to more accurately determine the actual construction costs. The estimates provided herein are based on 2020 approximations and provide on order of magnitude cost of the projects that will need to be refined and updated based on more accurate designs and construction cost index (CCI) increases. We also note that these estimates assume construction of the network as a single initiative. We note that fiber construction, like other construction projects, benefits from an economy of scale (including both the total length of fiber routes constructed, and a single mobilization by the construction contractor) and from a contractor that is focused on the project as its main mission. While we encourage the use of Dig Once opportunities and capitalizing on existing physical resources, we recommend that where possible, fiber construction be done as part of a large-scale fiber construction procurement, and not as a series of small projects done as peripheral parts of other utility projects. This is because of the high cost of planning, mobilization, and contracting for multiple small projects, as well as the likelihood of a larger project being more attractive to more competing construction contractors. A wide range of logical topologies are feasible given the physical architecture of the proposed network. Depending on splicing configurations, connections can be established with or without route diversity over backbone rings (where available), and can provide dedicated paths between any two sites without the need for “patching” between intermediate sites. The cost estimates are based on a flexible approach to splicing and fiber termination, providing a backbone consisting of a 144-strand cable occupying one 2-inch conduit. Factors That Affect Fiber Cost Estimates Beyond the physical fiber optic cable routing, a number of technical design and construction attributes affect costs, including the following: • Fiber strand count: The number of individual fiber strands provided in a single cable correlates to the capacity of the cable. Due to the vast effective bandwidth of City of Cupertino | Fiber Optic Master Plan | August 2020 37 fiber, it is feasible to scale the rate of data transmission carried by even a single fiber strand to meet all of the City’s needs indefinitely; however, the cost of network electronics increases exponentially with this capacity. On the other hand, the material cost of fiber strands represents a very minor component of the overall cost of fiber construction (about $0.01 per strand per foot, compared to $50 to $100 per foot for the total cost of typical construction). It is thus prudent to install a cable of sufficient size to meet any conceivable requirements to ensure these needs can be met with a configuration of electronics that are as low-cost as possible. In fact, with sufficient fiber strands, the City can increase network capacity many orders of magnitude above current levels with little or no change to its network electronics. While we anticipate no portion of the network will require more than a few dozen strands, cost estimates are based on the installation of a 144-count cable along most segments of the network. This will ensure sufficient capacity for nearly any conceivable expansion of internal needs, fiber leasing, or even future support of business or residential services for the foreseeable future. • Underground versus aerial construction: The cost estimates anticipate completely underground construction of the fiber, with fiber cables placed in a 2-inch conduit. Cost savings may potentially be possible by employing “aerial” construction (attaching fiber to utility poles). However, since the City does not own its own utility poles, aerial construction would require negotiating pole attachment agreements or joining the Northern California Joint Pole Association (NCJPA). Pole attachment generally requires recurring fees per pole, and generally require the attacher to pay the cost of any upgrades or modifications to the utility poles necessary to support the new attachment. NCJPA membership may reduce these fees but the City would still be required, as a new attacher, to cover the cost of moving the existing utilities and replacing poles. Our previous work with other cities in Northern California indicates that these “make ready” costs can be prohibitively expensive, usually in line with the underground costs, thus budgetary estimates incorporate only underground construction. • Conduit size and quantity: While it is possible to install fiber cable directly underground, this complicates installation and makes repairs difficult to City of Cupertino | Fiber Optic Master Plan | August 2020 38 implement without creating permanent impairments to the communications path. Instead, the cost estimates are based on the installation of a flexible plastic conduit that provides a path into which fiber cable can be installed, allowing for cable slack to be pulled to accommodate repairs, or for new cable to be installed to expand capacity. We assume underground construction will consist primarily of horizontal, directional drilling to minimize impact to the right-of-way (ROW) and to provide greater flexibility to navigate around other utilities. While cost estimates are based on the placement of a single 2-inch High-Density Polyethylene (HDPE) flexible conduit, it should be noted that placing additional conduits simultaneously results in relatively minor increases in cost, within limits. • Handhole placement and size: Handholes are enclosures installed underground in which conduit terminates. Handholes provide access to conduit for installing cable, as well as to house cable splice enclosures and cable slack loops required for future repairs. Handholes generally must be placed at intersections of multiple conduit paths, or where the conduit path makes a sharp change in direction. Handholes provide important access points to underground conduit, enabling expansion of the conduit infrastructure (i.e., installation of a lateral connection to a new network location) without disrupting conduit or installed cables. While cable can be pulled upwards of several thousand feet at a time, cost estimates for the City’s network assume installation of handholes every 500 feet on average, ensuring that the infrastructure supports cost-effective expansion to new sites, including access to businesses that might be targets of commercial network operators seeking to lease City fiber (or conduit space). • Right-of-way restoration and fees: The network cost estimates assume that the City may have to pay encroachment fees for construction along or under State roads and for railroad crossing application and licensing fees, which can total upwards of $15,000 per crossing, not including special construction costs, which generally entail steel encasement of conduit. The cost estimates assume that the City will incur typical costs for permanent asphalt and concrete restoration required following utility “test pitting” or “pot holing” (which is necessary to City of Cupertino | Fiber Optic Master Plan | August 2020 39 verify the location of, and prevent damage to, other utilities in the path of the fiber); test pitting generally consists of excavation within small areas of less than two feet in diameter. Outside Plant Construction Elements Outside plant (layer 1, also referred to as the physical layer) is both the most expensive part of the network and the longest lasting. The cost estimates are inclusive of all engineering, project management, quality assurance, and construction labor anticipated to be necessary to implement the network on a turnkey basis, and are based on relatively conservative pricing assumptions. The following summarizes the scope anticipated by each of the cost components itemized in the table above: • Engineering: Includes system level architecture planning, preliminary designs and engineering field walk-outs to determine candidate fiber routing; development of detailed engineering prints and preparation of permit applications; and post-construction “as-built” revisions to engineering design materials • Project Management / Quality Assurance: Includes expert quality assurance field review of final construction for acceptance, review of invoices, tracking progress, and coordination of field changes • General Outside Plant Construction: Consists of all labor and materials related to “typical” underground outside plant construction, including conduit placement, fiber installation, and surface restoration; includes all work area protection and traffic control measures inherent to all roadway construction activities • Railroad, Bridge, and Interstate Crossings: Consists of specialized engineering, permitting, and incremental construction (material and labor) costs associated with crossings of railroads, bridges, and interstate/controlled access highways • Outside Plant Fiber Splicing: Includes all labor related to fiber splicing of outdoor fiber optic cables • Fiber Termination/Building Entrance: Consists of all costs related to fiber lateral installation into network sites, including outside plant construction to the facility, building and or cabinet penetration, inside plant construction to a typical City of Cupertino | Fiber Optic Master Plan | August 2020 40 backbone network service “demarcation” point, fiber termination, and fiber testing. Actual costs may vary due to unknown factors, including: 1) costs of private easements, 2) congestion in the public right-of-way, 3) variations in labor and material costs, and 4) subsurface hard rock. Costs for underground placement were estimated using available unit cost data for materials and estimates on the labor costs for placing, pulling, and boring fiber based on construction in comparable markets and previous construction costs that the City has paid. The material costs were generally known apart from unknown economies of scale and inflation rates, and barring any sort of phenomenon restricting material availability and costs. For the low-cost estimate we used an underground construction cost per mile of $275,000, or roughly $50 per foot, which is similar to the costs SV-ITS paid for fiber construction. For the high-cost estimate we used $500,000 per mile, or almost $100 per foot, which matches what the City has seen on the higher end of some of its fiber optic construction projects. The City would be able to obtain more precise costs by completing detailed engineering and initiating through a competitive bidding process with fiber construction contractors. Network Maintenance Requirements In the sections below we offer guidance on the staff expertise and resources necessary for the City to maintain the planned network expansion. Our analysis and recommendations are based on detailed discussions with City staff and our expertise in fiber network construction and operations. In sum, we estimate the City would incur the following monthly expenses for operating and maintaining the existing and new fiber segments (Table 8). City of Cupertino | Fiber Optic Master Plan | August 2020 41 Table 8: Estimated Fiber Maintenance Costs Fiber Construction Phase Monthly Maintenance Costs Existing Fiber Network $6,200 Connect City Facilities and Redundancy to Core Sites $1,300 Connect Major Traffic Intersections $1,300 Connect City Parks and Downtown Wi-Fi $2,500 Redundancy to Fiber Network $1,000 Connect CCC Emergency Sites $1,000 Total $13,300 Fiber Maintenance and Repair Fiber maintenance and repair of outside cable plant involves tasks such as spicing fiber cable, replacing damaged conduit, and performing other fiber construction work using specialized equipment. This work is often required to be performed outside normal business hours, supported by on-call staff who are available 24/7/365. Fiber maintenance and repair also requires maintaining, storing, and having immediate access to any and all materials needed for network repair. Based on our discussions with City staff, we understand that the City’s resources dedicated to fiber maintenance are already at capacity and no formal process is in place for fiber maintenance or emergency repair. The addition of potentially 15 additional fiber miles will double the amount of required maintenance work. Thus, undertaking all of the fiber maintenance and repair responsibilities in-house would require a significant capital investment by the City. Further, the City would have to set up contracts with a company capable of providing the necessary maintenance, or the City would have to expand staffing and invest time to implement training. While the City could increase staffing to handle all fiber maintenance activities, we conclude that, at least initially, the City would be better served by hiring a contractor who would perform the actual maintenance and construction work. We note that this model is common among local governments. We recommend the City identify a fiber maintenance contractor to perform ongoing roles that include: City of Cupertino | Fiber Optic Master Plan | August 2020 42 • Implementing proper surveillance of critical infrastructure to identify failures • Ensuring provisions are in place to effect restoration activities • Performing network maintenance and fiber repairs as directed by the City • Warehousing network materials13 • Documenting incidents and outages • Maintaining processes to reduce the risk of damage to system components, and to periodically test their resiliency; and • Documenting the repair process and resolution of each incident and outage • Maintaining accurate and sufficiently detailed documentation to support all maintenance and restoration activities The annual cost of fiber maintenance and emergency repair is approximately $100,000, which includes the City’s existing fiber and all of the construction phases listed above. Costs are based on other contracting rates we have seen of $250 per month per mile of fiber network. The City’s relationship with the contractor should be covered by a fiber maintenance agreement that requires the contractor to resolve fiber outages and restore network services in a timely manner. As part of the agreement the selected contractor should commit to meeting the City’s required response times. If the contractor provides maintenance services for entities other than the City, it must commit to have adequate internal staff and resources to prioritize the City’s needs and meet the City’s requirements. The fiber maintenance agreement should clearly define the contractor’s requirements, including the following elements, needed to support the use cases in Section 4: Contractor emergency contact information – The contractor should provide emergency contact information and should be available on a 24x7x365 basis to receive notifications 13 While the City could opt to store materials, doing so could create inefficiencies in maintenance and emergency repair processes. In addition, most contractors already have storage space for materials—and experience maintaining inventories. City of Cupertino | Fiber Optic Master Plan | August 2020 43 by telephone, email, and/or any other electronic means requested by the City. The contractor should designate a single point of contact who should have primary responsibility for addressing maintenance-related issues concerning fiber assets and associated property. Scheduled maintenance – The contractor should perform scheduled maintenance work inclusive of repairs, network additions, and relocation of existing fiber infrastructure, at the direction of the City. Scheduled maintenance should be coordinated with the City. The contractor should notify the City to confirm the date of the scheduled work at least five business days in advance. Planned maintenance work should be scheduled after midnight and before 6:00 a.m. local time to reduce the potential impact to customers and City operations in the event of service outages caused by the maintenance work. Emergency repair – For any emergency repair of the City’s fiber network, the contractor should provide a guaranteed response of four hours (i.e., one-hour call back, and being on site with all required equipment no later than three hours after declaration of emergency). A four-hour response time is common in a metropolitan area such as Cupertino. Warehousing of spare network materials – Repairs may include replacing a run of conduit between two or more handholes, replacing several thousand feet of fiber cable, or splicing fiber. The contractor should own and store a reasonable quantity of any and all materials required for emergency repairs to fix damaged fiber and restore service to the City’s network. Due to its nature, scheduled maintenance work should not require storage of materials as it can be planned with enough lead time for the contractor to procure the required materials. The most common type of emergency maintenance is repairing of cable cuts and breaks. This effort typically requires fiber splicing and occasionally involves placement of new conduit and fiber cable. Thus, the key materials to be warehoused will include fiber optic cable and splice enclosures matching each variety deployed on the network, as well as 2- inch HDPE conduit (which has been specified through the network). Other items include handholes, ground rods and other miscellaneous construction materials but these items will generally be needed less frequently. City of Cupertino | Fiber Optic Master Plan | August 2020 44 Stored materials expended during emergency repair should be replenished immediately to ensure availability for future incidents. However, at any one time the contractor should have on hand enough stored material to account for several incidents at a time. Documentation of maintenance work – The contractor should be required to provide redlined drawings to the City upon completion of repairs, relocations, or additions of fiber infrastructure. Documentation should include high-quality digital pictures of the fiber failure, and the names and addresses of the party(s) that caused the fiber failure (if applicable). For each assignment, the contractor should provide the City with updated as-built drawings and an incident report. Additionally, the contractor should track the hours and materials required to repair the fiber and provide the City with this information at the time of submitting an invoice for work performed. Underground Utility Locates Preventing damage to outside cable plant is an important part of an overall strategy to mitigate the risks of physical network outages. An efficient, thorough, and timely locating process can help prevent damage to the infrastructure. Locating fiber optic plant accurately and quickly in response to utility locate requests (i.e., USA North “811” tickets14) helps ensure that construction activities in the City do not result in accidental damage. Whether performed by internal staff or contractors, processes must be in place to perform locates in accordance with applicable damage- prevention laws. Maintaining accurate documentation of the physical plant can help locators perform this function more accurately; when maintained electronically in a GIS format using a fiber management software, the documentation can be included in the construction plans for other capital projects more readily to facilitate proactive avoidance of potentially vulnerable fiber. The City currently performs its own locates for fiber. A transfer of knowledge or oversight from current personnel may be required as the City scales its locate capacity to accommodate the 16 miles of new fiber infrastructure. For a contractor-provided solution, we estimate a cost of approximately $150 per mile of locatable infrastructure per month; 14 “811 in Your State: California,” https://call811.com/811-In-Your-State/Map/State/California (accessed April 6, 2020). City of Cupertino | Fiber Optic Master Plan | August 2020 45 based on that estimate, the existing 13 miles and the new 18 miles of fiber would require an annual contractor expense of about $50,000. 5.2 Alternative Approach: Leveraging Existing-Third Party Provider’s Fiber to Meet Current and Future City Needs An alternate approach to constructing its own fiber would be for the City to lease dark fiber from a third-party provider that has assets in the City. In particular, Crown Castle has fiber in proximity to schools, since it is the fiber provider for the school districts. Zayo and Verizon also have fiber in the vicinity. A long-term lease, such as a 20-year indefeasible right of use (IRU) agreement, which is the norm for the industry, might eliminate the City’s need to construct, maintain, or repair the fiber required to expand fiber to some of the City’s buildings and other locations. Leasing dark fiber may not provide savings, however, if the City still has to pay for construction of fiber on a large percentage of a given route (i.e., if no provider has dark fiber along all or most of the routes the City requires). If the City is interested in analyzing dark fiber as an option, it should first consider an RFI process by which it would gather detailed input from interested carriers. Dark Fiber IRU Construction and Lease Costs Market considerations will drive the cost that a third-party provider would charge the City for a dark fiber IRU. These considerations can include the level of competition in the market, the provider’s availability or scarcity within their existing fiber network, and the demand for dark fiber from other entities. The norm in the industry is for the fiber lease customer to pay the cost of any needed fiber construction from the provider’s network to the new site in a lump sum, then pay a lease cost for the fiber strands already in place. The following table lists the estimated minimum lump sum and monthly market rates a third-party provider would charge the City for fiber lese for each fiber construction phase. City of Cupertino | Fiber Optic Master Plan | August 2020 46 Table 9: Estimated Dark Fiber IRU Pricing for Fiber Lease Phase Minimum IRU Lump Sum Cost Monthly IRU Cost (Market Rate) Connect City Facilities and Redundancy to Core Sites $1.0 million $3,400 Connect Major Traffic Intersections $830,000 $5,200 Connect City Parks and Downtown Wi-Fi $1.5 million $9,000 Connect CCC Emergency Sites $390,000 $4,600 Total $3.7 million $22,200 We estimated the market rate cost of the fiber connections to be $150 per month per strand per mile based on data from other urban markets. Actual pricing would need to be obtained from third-party providers (possibly in a competitive bid process). Fiber routing and lateral construction miles (Table 10) were based on data available from existing third-party providers as well as our understanding that the major fiber routes through the City parallel the larger traffic thoroughfares. We also assumed that the third- party providers can construct fiber at the low end of estimated costs in City could construct fiber—$275,000 per mile. Table 10 also provides, for comparison, the estimated miles of fiber construction for the City-constructed scenario. In most cases, the IRU is not a sound approach, because it requires almost as much new construction of fiber (and hence, cost) as the City-constructed approach. Table 10: Estimated Dark Fiber Mileage Required Phase Estimated Miles of New Dark Fiber Construction Estimated Miles of Existing Dark Fiber Estimated Miles of City-Owned Fiber Construction Connect City Facilities/ Redundancy to Core Sites 3.0 7.5 3.3 Connect Major Traffic Intersections 3.3 14.2 1.5 Connect City Parks and Downtown Wi-Fi 6.2 24.9 5.7 Connect CCC Emergency Sites 2.5 14.2 2.5 Total 15.0 60.8 13.0 City of Cupertino | Fiber Optic Master Plan | August 2020 47 For example, connecting the five City facilities back to City Hall would require an estimated 3 miles of new fiber construction and 7.5 miles of existing provider fiber. (By comparison, the City would only need to construct 3.3 miles of fiber to achieve the same objective.) The estimate includes redundant paths back into City Hall from the provider network. The IRU would require a lump sum of at least approximately $1.0 million to recover the cost of the fiber construction. A fiber provider may mark up this fee based on market rate and other profitability factors such a project management costs for the fiber construction. Dark Fiber Would Deliver Less Functionality Than City-Owned Fiber Relative to City-owned fiber, leased dark fiber has technical disadvantages. For example, a government-owned fiber network is completely under the government’s control and all routes and access points are known. In a lease arrangement, we recommend the City have access to the physical routing of each of the dark fiber services and knowledge of the maintenance contracts that the provider has in place in the event of a fiber failure on the routes. These can be requirements of a competitive bid for dark fiber services. Dark fiber leasing also limits the number of strands of fiber available at each site. While the City could theoretically lease as many strands as it needs, in practice, doubling the strands doubles the monthly lease costs. While technologies such as wave division multiplexing can be used to decrease the number of strands needed, addressing fiber scarcity with electronics increases the cost of the network electronics and reduces the City’s flexibility in managing its network. In its existing fiber network, the City has flexibility to allocate fiber strands as needed. This would allow the City to segment portions of the network off to different fiber strands, such as a pair of fiber for public Wi- Fi and a pair of fiber for internal City networking. It also enables the City to use its spare capacity to connect to new sites. Government-owned fiber allows the City to make changes and perform maintenance on its own timeline whereas dark fiber leasing requires the third party provider to make changes on their schedule according to contract terms. Fiber changes and maintenance on shared fiber routes will also require coordination with the other fiber lessees before maintained or changes can be made, potentially slowing down the process. IRUs typically come with SLAs that guarantee uptime with a financial compensation to the lessee in the form of a reduced monthly fee that month. However, if the fiber service goes down and critical government networks do not work, the typical level of financial compensation City of Cupertino | Fiber Optic Master Plan | August 2020 48 rarely makes up for the lack of service. We recommend in this case that the City maintain backup leased services for mission-critical sites. 5.3 Summary of Estimated Fiber Expansion Costs The following table compares the estimated costs of the City constructing fiber to the City’s potential dark fiber IRU costs (lump sum and monthly recurring costs). Network electronic costs ($312,800) would be the same for both models, as described in the next section. City of Cupertino | Fiber Optic Master Plan | August 2020 49 Table 11: Total Estimated Fiber Expansion Costs Fiber Construction Phase City-Owned Fiber Costs15 Dark Fiber Lease Costs Construction Cost Estimate City-Owned Monthly Maintenance Costs 10-Year Total Cost of Ownership Estimated Minimum IRU Lump Sum Cost Estimated Monthly IRU Cost (Market Rate) 10-Year Total Cost of Ownership Connect City Facilities and Redundancy to Core Sites $900,000 $1,300 $1,056,000 $1.0 million $3,400 $1.4 million Connect Major Traffic Intersections $880,000 $1,300 $1,036,000 $830,000 $5,200 $1.5 million Connect City Parks and Downtown Wi- Fi $1.7 million $2,500 $2.0 million $1.5 million $9,000 $2.6 million Redundancy to Fiber Network $640,000 $1,000 $760,000 N/A N/A N/A Connect CCC Emergency Sites $700,000 $1,000 $820,000 $390,000 $4,600 $942,000 Total $4.8 million $7,100 $5.7 million $3.7 million $22,200 $6.4 million 15 City costs assume economies of scale from large projects. City of Cupertino | Fiber Optic Master Plan | August 2020 50 5.4 Estimated Network Electronics Costs Would Be the Same for City- Owned Fiber and Dark Fiber The following sections outline the costs of network electronics required to operate the City’s expanded fiber segments. These costs would be the same whether the City constructs its own fiber or leases dark fiber. The total estimated capital cost for electronics to operate all fiber segments, including to CCC emergency sites, is $831,600 (Table 12). Table 12: Estimated Network Electronics Costs for New Fiber Segments Phase Additional Network Equipment Costs Existing Fiber Network N/A Connect City Facilities and Redundancy to Core Sites $5,000 Connect Major Traffic Intersections $49,000 Connect City Parks and Downtown Wi-Fi $105,600 Redundant Network Cores $87,000 Connect CCC Emergency Sites $585,000 Total $831,600 Electronics to Operate Fiber to City Facilities The existing network switches can be used to support the fiber expansion. The City would need to install fiber optic media converters or small form-factor pluggables (SFP) into the switch at the site and the core switch at City Hall. The site could then migrate from the Comcast connection to the fiber. We estimate this will cost $5,000, as outlined in the following table: Table 13: Estimated Network Electronic Costs for Fiber to City Facilities Sites Quantity Cost Subtotal 1 Gbps SFPs (Both Sides) 10 $500 $5,000 Electronics to Operate Fiber to Traffic Intersection New rugged switches will need to be purchased to connect the traffic signal cabinets and connect them to the transportation network. Rugged switches are required to withstand the temperatures and humidity in the traffic signal cabinets and support power over Ethernet to power cameras or other Smart City devices at the traffic intersections. SFPs are also needed to connect the rugged switch to the transportation network over the fiber. City of Cupertino | Fiber Optic Master Plan | August 2020 51 We estimate the cost will be $49,000. The following table outlines the costs. Table 14: Estimated Network Electronic Costs for Fiber to Traffic Intersections Intersections Quantity Cost Subtotal Rugged Switch 14 $2,500 $30,000 1 Gbps SFPs (Both Sides) 28 $500 $12,000 Total $49,000 Electronics to Operate Fiber to City Parks and Downtown Wi-Fi New rugged switches will be needed to connect the City Parks and the downtown area to the core switch at City Hall. Rugged switches are required to withstand the temperatures and humidity of being outdoors and should support power over Ethernet to power the wireless access points. A weatherized enclosure will be required to house the switch and fiber termination. We envision the switch will be installed on an existing light pole to support the deployment of Wi-Fi in the parks. The three sector wireless access points would be installed further up on the light pole. We estimate the cost to be $105,600. The following table outlines the costs. Table 15: Estimated Network Electronics Costs for Fiber to City Parks and Downtown Wi-Fi Parks Quantity Cost Subtotal Rugged Switch 16 $2,500 $40,000 1 Gbps SFPs (Both Sides) 34 $500 $16,000 NEPA Enclosure 16 $100 $1,600 Wireless Access Point 48 $1,000 $48,000 Total $105,600 The City could also elect to install Smart City poles in the park rather than use an existing light pole. Smart City poles can support a variety of applications such as: • Wi-Fi • Digital signage • Video surveillance • Emergency call box • Small cell siting • Environmental sensors Figure 16 is a depiction of a smart pole. City of Cupertino | Fiber Optic Master Plan | August 2020 52 Figure 16:Smart Pole Depiction We estimate the installation cost of a full smart pole to be $50,000 to $115,000 based on our previous work in San Francisco. The following table outlines the costs for adding smart poles at the City’s parks. The poles can be viewed both as value-added to the park (and therefore part of a Park and Recreation initiative) and as a broadband and information technology initiative. The higher-end estimate reflects the most feature-laden approach and pricing for one-off equipment. Lower costs can be obtained if the City purchases poles on a larger scale. City of Cupertino | Fiber Optic Master Plan | August 2020 53 About half the cost is installation, which can be reduced if the device is installed as part of a capital project in the same area, or as a partnership with a wireless provider or other entity seeking to place infrastructure in the park or in the right-of-way. Table 16: Estimated Costs for Adding Smart Poles at City Parks and in Downtown Area Parks Quantity Low Cost High Cost Low Subtotal High Subtotal Smart Poles 16 $50,000 $115,000 $800,000 $1,140,000 Electronics to Operate Fiber to CCC Emergency Sites Because the CCC emergency sites are primarily used as parking lots where utility power may not be available (especially during an emergency), the network electronic solution needs to have independent power and ideally not use a generator, which requires maintenance and fuel. One solution is a variation on the smart pole in Section 5.4.3. Fiber would be installed to the smart pole, which has an added solar panel and battery backup. In the equipment enclosure, fiber is terminated and connected to an Ethernet switch connected through Power over Ethernet (PoE) to wireless access points. The wireless access points can provide Wi-Fi to residents during an emergency and can be used by emergency personnel. Emergency personnel can also connect their equipment directly to the Ethernet switch for network connectivity. The equipment could be connected to a generator on a trailer to provide supplemental power if necessary. Figure 17 illustrates the CCC emergency smart pole concept. City of Cupertino | Fiber Optic Master Plan | August 2020 54 Figure 17: CCC Off-Grid Solar-Powered Smart Pole We estimate the cost for the CCC off-grid solar-powered smart poles to be approximately $600,000. The following table outlines the cost detail. City of Cupertino | Fiber Optic Master Plan | August 2020 55 Table 17: CCC Smart Pole Cost Estimates Equipment Quantity Cost Subtotal Rugged Switch 6 $2,500 $15,000 1 Gbps SFPs (for Both Sides) 12 $500 $6,000 Enclosure 6 $1,000 $6,000 Wireless Access Point 18 $1,000 $18,000 Smart Pole with Camera and Charging Station 6 $75,000 $450,000 Off-Grid Solar Power System 6 $15,000 $90,000 Total $585,000 Electronics to Operate Fiber Providing City Network Redundancy If the City constructs its own fiber, the City can splice fiber so that a second core can be deployed at the service center with redundant connections from each remote site back to both City Hall and the Service Center cores. By also connecting the two cores in a ring, the City could increase the network redundancy and resiliency. The City already operates the open shortest path first (OSPF) routing protocol which would allow the network to fail over to the redundant connections. The service center can also have a backup Internet connection in case the primary fails at City Hall. The City would need to purchase a modular core switch for the Service Center to serve as the second core. The new core would be connected to City Hall in a 10 Gbps ring. Additional 10 Gbps links can be added as network capacity grows. Once the core ring is established the remote sites can be connected to both cores by adding SFPs at the new core and the remote site switches. The cost for implementing network redundancy is $87,000. The following table outlines the costs. Table 18: Estimated Network Electronics Costs for Fiber Segments to Increase Redundancy Second Core at Service Center Quantity Cost Subtotal Modular Core Switch 1 $45,000 $45,000 I Gbps SFPs for Redundant Connections 44 $500 $22,000 10 Gbps SFPs to City Hall 4 $5,000 $20,000 Total $87,000 City of Cupertino | Fiber Optic Master Plan | August 2020 56 6 Potential Fiber Network Business Plans We considered potential business models identified by the City: 1. The City owns and operates the fiber network 2. The City permanently outsources operation and management of the network (with City or third-party ownership)—referred to below as a “commercial approach” 3. A hybrid approach that enables the City to trade or lease some excess fiber, outsource some maintenance and operations, and pursue “Dig Once” opportunities with City capital projects and excavation by non-City entities 6.1 City-Owned and Operated Fiber Network The City has successfully owned and operated its fiber network for more than 10 years— and has created real value for the City. Specifically, has City has offset the cost of leased circuits in connecting buildings and traffic infrastructure. It has also provided more capacity, at a higher level of reliability, than the leased service. The fiber appears to be in good condition, with many years of useful life and the ability to operate at thousands of times the speeds of current electronics, without modification. The City early on used best practices in funding and obtaining value from the network: 1. The City leveraged grant funding—almost all construction cost was covered by federal and state funds 2. The City partnered with SV-ITS and Santa Clara County, which ensured the network excavation created value for a wide range of stakeholders 3. The City recognized it could use the fiber both for transportation communications and other City government needs However, there are areas where practices could be improved to create even more value and efficiency. First, the network would be more valuable and reliable if the City had a contract in place to repair fiber outside plant damage. Having a contract would increase the City’s ability to rely on applications that are in the cloud or centralized at City Hall or the disaster recovery facility and would increase the reliability of network connections between buildings and the outside. The increased reliability could also be valuable to potential external stakeholders (such as wireless service providers or large businesses or institutions) that the City may wish to connect. City of Cupertino | Fiber Optic Master Plan | August 2020 57 Second, there is some (but not an overwhelming amount of) excess capacity that the City may wish to consider leasing or trading. There are examples of city and state governments that have bartered strands from their excess capacity and significantly expanded their footprints. One possibility would be to trade excess fiber strands on arterial routes (where most City fiber is located) for fibers to sites in neighborhoods near parks and schools (where Crown Castle has previously constructed fiber). Third, the cost of fiber construction in the City is high. This is for several reasons, including that: 1. Construction would either need to be in crowded rights-of-way on busy arterials or on crowded utility poles 2. Labor costs and restoration costs are high 3. The City does not typically allow use of microtrenching construction techniques that could reduce construction costs due to enhanced risks to existing utilities and increased vulnerabilities to damage As a result, while there is sufficient fiber in the existing network to add new sites, the “last mile” to the new sites may cost hundreds of thousands of dollars, and connecting with new fiber may not always be a cost-effective approach. 6.2 Full Commercial Approach It is not clear that a fully commercial approach would address these challenges or open new opportunities. A full commercial approach would have the advantage of enabling an entity specializing in commercial broadband to monetize the fiber and maintain it. One scenario would be that the City would sell its fiber or provide a long-term lease to an entity to maintain the fiber and sell and operate the unused fiber. The City would stipulate which fiber strands it would keep under what terms, as well as the terms of maintenance and future use capability. The City may obtain revenue or a one-time windfall in transferring the asset. Based on the cost to construct the fiber (in the $5 million range), the City’s one-third interest, at most, would be less than $2 million, and probably significantly less given that many strands are used. As an illustration, lease costs in comparable markets are approximately $1,800 per mile, per strand, per year. A fully commercial approach would be technically challenging for many reasons. One is that the fiber is in a shared cable (and passes through shared conduit and manholes). If the SV-ITS parties do not participate in the arrangement, the commercial entity would City of Cupertino | Fiber Optic Master Plan | August 2020 58 only have access to the City’s allotment, and would need to provide the existing terms of access and maintenance to those parties. Second, the excess fiber count is limited. Depending on the route, one-half to one-third of the City’s fiber is available—approximately 10 to 16 strands, which taken together may only connect a handful of additional locations and provide limited value. Third, while there are not explicit exclusions on reselling or leasing the City’s fiber, the SV-ITS agreements do specify that the City fiber is for City use. Furthermore there may be exclusions in the funding that may require clarifications and renegotiations with the grantors, including U.S. DOT (FHWA) and Caltrans. Fourth, while there is not a large reservoir of excess fiber, the excess fiber that exists may provide significant value to the City for connecting new locations. Fifth, it is not clear what competitive value the City fiber would have for a third-party, given competition from other providers over many of the same arterial routes. The maximum value would be the construction value of the fiber, but the market value may be significantly less, for all of the above reasons. 6.3 Hybrid Approach The City can realize some of the advantages of commercializing the fiber by adopting a mixed approach. Whether or not the City commercializes the fiber, an on-call contract vehicle should be considered to increase reliability. Once one is in place, we recommend a business approach that: • Trades excess fiber for strands the City wants • Offers fiber for lease if the City believes it does not need the excess capacity on a route and is technically able to commit to commercial performance standards • Develops a Dig Once policy (Section 7) to expand fiber cost-effectively if new opportunities come from new construction—either by the City or by other communications providers and utilities • Incorporates fiber build cost in City capital projects such as new buildings—so that fiber location becomes a factor in facilities location Appendix C provides a list of technical and business activities the City would need to undertake prior to leasing its fiber. City of Cupertino | Fiber Optic Master Plan | August 2020 59 7 Dig Once Policy Recommendation Based on our review of the City’s network and processes, we prepared Dig Once suggested build criteria, coordination process recommendations, overview specifications, and process recommendations. 7.1 Criteria We recommend developing criteria that would lead the City to select an excavation opportunity for Dig Once conduit. Dig Once is never simply “throwing extra conduit or fiber into a hole”—at a minimum, adding more conduit or fiber on a project requires designing the extra facilities, acquiring more materials, additional labor, sizing manholes and handholes appropriately, determining how all entities access and maintain the infrastructure, making sure all clearances and spacing are correct, testing and documenting what is built. That said, especially in areas with expensive construction costs and congestion, like Cupertino, a well-implemented Dig Once opportunity can result in savings from 50 to 90 percent relative to new construction. The following criteria are consistent with good practices. We recommend giving potential projects scores for each of the criteria: 1. The length of the project (i.e., ideally, select projects that will result in a long continuous route offering a wide range of potential uses in the future. All other things being equal, a quarter-mile or more provides significantly more value than projects that are a block or less and is a suitable starting point as a cut-off— although very high cost routes (major street crossings, bridges) may provide value, even though they are very short. 2. The proximity of the project to current to or planned City facilities and/or community locations requiring service (i.e. health clinics, hospitals, areas of economic development) 3. The presence of existing City communications infrastructure in the vicinity of the project (in which case new conduit might not be needed) 4. The opportunity to use the project to cost-effectively build across physical constraints (e.g., bridges, freeway underpasses) 5. The interest of known partners or customers willing to lease conduit or fiber from the City so as to defray the build costs City of Cupertino | Fiber Optic Master Plan | August 2020 60 6. The lack of alternative options, due to crowded utility poles or congestion in the right-of-way, in the same corridor 7.2 Coordination with Excavators Coordinating with excavators—potentially through quarterly outreach or filing requirements—is an important best practice, even if only at first City excavators and builders are engaged. This is best facilitated by creating a working group of telecommunications providers, utilities and large builders in the City. The earlier opportunities are known, the earlier they can be considered for Dig Once. That enables (but does not guarantee) more coordination among excavators, earlier engineering, and lower costs. We recommend the City consider notifying non-City excavators as soon as possible of excavation projects where they may be able to take advantage of joint trenching. It is generally not sufficient, for example to only offer a 30-day window—and most City capital projects are planned well in advance of construction. 7.3 Overview Specifications We recommend the City develop overview specifications that provide the physical requirements for additional City conduit to be placed in tandem with a construction project. We suggest that the City require that City handholes be used for easy identification of handholes built for the City. Labeling the covers with the owner’s name will help reduce problems with locates, repair, and abandonment. In general, two 2-inch conduit should suffice. While 1-1/4” conduit have space for almost any size fiber cable, two-inch conduit provide flexibility to place two cables if needed. We recommend that for crossing major intersections, such as Interstate 280, the City consider a specification of four 2-inch conduit. The City can either develop additional or broader specifications or express a willingness to work with the excavator on an approach suited to its project. Figure 18 is a typical diagram showing Dig Once coordination in a major intersection corridor with a communications excavator. City of Cupertino | Fiber Optic Master Plan | August 2020 61 Figure 19 is a typical diagram showing Dig Once coordination with a water, power or sewer excavator, in a major intersection corridor and provides two options—one with Dig Once conduit directly above the utility, and one with Dig Once conduit offset laterally. Figure 20 is a typical diagram showing Dig Once coordination with a communications excavator. Figure 21 is a typical diagram showing Dig Once coordination with a water, power or sewer excavator, and provides two options—one with Dig Once conduit directly above the utility, and one with Dig Once conduit offset laterally. Typical drawings contain recommended standards for depth, bend, location, location tape, and vaults/handholes. While the most pronounced savings from Dig Once occur when the primary excavator is opening a trench or building a new road or sidewalk, economies are also possible in a project where the excavator is using directional boring. Generally speaking, directional boring in a range from two up to six inches in diameter provide economies in adding conduit, relative to a new standalone project. In other words, if the primary project calls for two, 2-inch conduit (as in Figure 20), there generally will be savings in adding two more conduit, relative to a new standalone project. City of Cupertino | Fiber Optic Master Plan | August 2020 62 Figure 18: Typical Diagram – Major Corridor – Dig Once Coordination with a Communications Excavator City of Cupertino | Fiber Optic Master Plan | August 2020 63 Figure 19: Typical Diagram – Major Corridor – Dig Once Coordination with a Water, Power, or Sewer Excavator City of Cupertino | Fiber Optic Master Plan | August 2020 64 Figure 20: Typical Diagram – Dig Once Coordination with a Communications Excavator City of Cupertino | Fiber Optic Master Plan | August 2020 65 Figure 21: Typical Diagram – Dig Once Coordination with a Water, Power, or Sewer Excavator City of Cupertino | Fiber Optic Master Plan | August 2020 66 7.4 Process We recommend developing a procedure for reviewing and accepting Dig Once infrastructure, including requirement for the Dig Once excavator to provide as-built documentation. We provide high-level process flow charts in Figure 22 and Figure 23. Again, ideally the project should be considered as far in advance as possible—part of a quarterly or monthly coordination meeting with excavators, rather than close to submittal of permits, in order for the excavator to understand well in advance if the City wants to participate. The informal pre-planning process the City undertakes with wireless providers may be a suitable model. The excavators responsible for any permitted infrastructure, even if not Dig Once, should be required to submit final Google Earth KML files so that the City can maintain an inventory of broadband infrastructure options, whether for City use or for access by the public. City of Cupertino | Fiber Optic Master Plan | August 2020 67 Figure 22: High-Level Dig Once Process Flowchart – Project Review City of Cupertino | Fiber Optic Master Plan | August 2020 68 Figure 23: High-Level Dig Once Process Flowchart – Implementation City of Cupertino | Fiber Optic Master Plan | August 2020 69 7.5 Bidding Consideration If the City selects a City excavation project as a Dig Once opportunity, we recommend including the Dig Once infrastructure in the base bid. We have seen, in other cities, instances in which the winner of the base bid has put unreasonable prices on an optional communications conduit—either to gain an unreasonable margin on the optional task or to change the City’s decision on Dig Once, so as to avoid doing the optional task. For example, the City of Santa Cruz Water Department shared its experience that this is a common bidding technique for their bids—which implies that Dig Once for the public sector might be considered more as an opportunity for Public Works to coordinate roadway remediation. (“Pave Once” might be a more apt name for the process.) It is also important to coordinate Dig Once into early part of engineering; adding the conduit as a change order also makes the conduit unreasonably expensive. 7.6 Construction Requirements We recommend that the City suggest ways in which it might incentivize sizing conduits or adding conduit for excess capacity. These might include offering favorable terms in master agreements, such as use of government buildings or properties for hub facilities, or reduced costs for placement of infrastructure on City light poles. 7.7 Leadership and Process Considerations It is important that the City’s process be appropriately staffed. The percentage of full-time equivalent (FTE) staff will depend on the number of potential projects and will need to be estimated based on the number of potentially suitable excavations. Based on discussions with City staff, there are no suitable capital projects planned at the moment, so this estimate will be based on the volume of projects by other utilities and providers. In a large-scale approach including Dig Once, fiber construction and fiber leasing, staff will need to handle Dig Once coordination and quality control, while also overseeing City fiber builds, marketing City fiber/conduit to potential partners, coordinating applications by commercial wireless providers, and coordinating with Santa Clara County, the water service providers, Cupertino Sanitary District, and neighboring communities. City of Cupertino | Fiber Optic Master Plan | August 2020 70 Appendix A: City Fiber Network Applications City of Cupertino | Fiber Optic Master Plan | August 2020 71 Appendix B: Needs Assessment Interviews CTC met with the following City staff: Department of Innovation and Technology • Bill Mitchell, Chief Technology Officer • Benny Hsieh, Infrastructure Manager • Peter Coglianese, Multimedia Communications Specialist Department of Public Works • Chad Mosley, P.E., Assistant Director and City Engineer • Steve Pagan, P.E., Associate Engineer • Jennifer Chu, P.E., Senior Civil Engineer • David Stillman, Transportation Manager Cupertino Amateur Radio Emergency Service (CARES) • Marcel Stieber, Assistant Emergency Coordinator Technology, Information and Communication Commission • Mukesh Garg, Commissioner • Naidu Bollineni, Commissioner City of Cupertino | Fiber Optic Master Plan | August 2020 72 Appendix C: Fiber Lease Checklist Table 19: Technical and Business Activities for City Fiber Lease Phase Track Technical Business Planning Identify fiber routes for lease • Existing routes • Planned extensions • Facilities (substations, splice points, rack space) Develop service catalog • Fiber performance options • Dark fiber strands (in existing bundle) • Dedicated fiber cable (new or existing cable) • Fiber monitoring and alerting Develop fiber maintenance and replacement cost estimates Create fiber pricing models (replacement cost approach, income approach, market rates) Design parameters aligned with service catalog metrics • Metro / regional single mode (i.e. G.652.D) • Splice loss tolerance (define allocations for express and distribution tubes and/or cables) Develop agreement templates • Long-term IRU • Monthly leases Create specifications for customer tie points Create SLA technical parameters (repair response times, customer access timeframes / procedures, fiber performance) Create SLA templates Implementation Develop construction specifications (outside plant, fiber termination / customer tie points) Develop construction contracts Develop construction plans and permits (including splice matrices) for new construction Obtain easements (if applicable for laterals and/or new segments) Develop fiber management platform / GIS systems Operations and Support Create specifications for customer tie points Develop fiber maintenance and repair contracts Create fiber support material supply list / stockpile Develop help desk / customer support contracts Create as-built package template • Fiber route maps / GIS data • Fiber strand allocations • Fiber characterization / performance test data Develop escalation procedures for repairs Create customer notification processes and contact lists Develop monitoring and NOC service contract (if applicable) City of Cupertino | Fiber Optic Master Plan | August 2020 73 Appendix D: Proposed Future Routes City of Cupertino | Fiber Optic Master Plan | August 2020 74 Appendix E: Fiber Count and Ownership