ITEM-3 C16-2022-0004 ADV PACKET PART1 — original pdf
Backup
CITY OF AUSTIN Board of Adjustment Decision Sheet C-2 DATE: July 11, 2022 CASE NUMBER: C16-2022-0004 _______Thomas Ates _______Brooke Bailey _______Jessica Cohen _______Melissa Hawthorne _______Barbara Mcarthur _______Ryan Nill _______Darryl Pruett _______Agustina Rodriguez OUT _______Richard Smith OUT _______Michael Von Ohlen _______Nicholl Wade _______Kelly Blume (Alternate) N/A _______Carrie Waller (Alternate) _______Marcel Gutierrez-Garza (Alternate) OWNER/APPLICANT: Apple Tree Holdings LLC-Tony Nguyen ADDRESS: 4507 IH 35 SVRD NB VARIANCE REQUESTED: The applicant is requesting a sign variance(s) from the Land Development Code, Section 25-10-123 (Expressway Corridor Sign District Regulations): (B) (2) (b) (i) to exceed sign area of 107 square feet (maximum allowed) to 200 square a) feet (requested) and (B) (3) (a) to exceed sign height of 35 feet (maximum allowed) to 50 feet (requested) in b) order to provide signage for a multi-tenant professional office in a “LO-NP”, Limited Office- Neighborhood Plan zoning district. (Upper Boggy Creek Neighborhood Plan) Note: The Land Development Code sign regulations 25-10-123 Expressway Corridor Sign Regulations (B) This subsection prescribes regulations for freestanding signs. (1) One freestanding sign is permitted on a lot. Additional freestanding signs may be permitted under Section 25-10-131 (Additional Freestanding Signs Permitted). (2) The sign area may not exceed: (a)on a lot with not more than 86 linear feet of street frontage, 60 square feet; or (b) on a lot width more than 86 linear feet of street frontage, the lesser of: (i) 0.7 square feet for each linear foot of street frontage; or (ii) 300 square feet. Item-3/1 (B) This subsection prescribes regulations for freestanding signs. (3)The sign height may not exceed the greater of: (a) 35 feet above frontage street pavement grade; or (b) 20 feet above grade at the base of the sign. BOARD’S DECISION: BOA MEETING JUNE 13, 2022 POSTPONED TO JULY 11, 2022 (AE DENIAL); JULY 11, 2022 POSTPONED TO AUGUST 8, 2022 BY BOARD (AE DENIAL) FINDING: 1. The variance is necessary because strict enforcement of the Article prohibits and reasonable opportunity to provide adequate signs on the site, considering the unique features of a site such as its dimensions, landscape, or topography, because: OR, 2. The granting of this variance will not have a substantially adverse impact upon neighboring properties, because: OR, 3. The granting of this variance will not substantially conflict with the stated purposes of this sign ordinance, because: AND, 4. Granting a variance would not provide the applicant with a special privilege not enjoyed by others similarly situated or potentially similarly situated, because: ______________________________ Elaine Ramirez Executive Liaison Jessica Cohen Madam Chair ____________________________ forItem-3/2 CITY OF AUSTIN Board of Adjustment Decision Sheet C-1 DATE: June 13, 2022 CASE NUMBER: C16-2022-0004 _______Thomas Ates _______Brooke Bailey _______Jessica Cohen _______Melissa Hawthorne OUT _______Barbara Mcarthur _______Darryl Pruett _______Agustina Rodriguez _______Richard Smith _______Michael Von Ohlen _______Nicholl Wade _______Kelly Blume (Alternate) _______Carrie Waller (Alternate) _______Marcel Gutierrez-Garza (Alternate) OWNER/APPLICANT: Apple Tree Holdings LLC-Tony Nguyen ADDRESS: 4507 IH 35 SVRD NB VARIANCE REQUESTED: The applicant is requesting a sign variance(s) from the Land Development Code, Section 25-10-123 (Expressway Corridor Sign District Regulations): (B) (2) (b) (i) to exceed sign area of 107 square feet (maximum allowed) to 200 square a) feet (requested) and (B) (3) (a) to exceed sign height of 35 feet (maximum allowed) to 50 feet (requested) in b) order to provide signage for a multi-tenant professional office in a “LO-NP”, Limited Office- Neighborhood Plan zoning district. (Upper Boggy Creek Neighborhood Plan) Note: The Land Development Code sign regulations 25-10-123 Expressway Corridor Sign Regulations (B) This subsection prescribes regulations for freestanding signs. (1) One freestanding sign is permitted on a lot. Additional freestanding signs may be permitted under Section 25-10-131 (Additional Freestanding Signs Permitted). (2) The sign area may not exceed: (a)on a lot with not more than 86 linear feet of street frontage, 60 square feet; or (b) on a lot width more than 86 linear feet of street frontage, the lesser of: (i) 0.7 square feet for each linear foot of street frontage; or (ii) 300 square feet. (B) This subsection prescribes regulations for freestanding signs. (3)The sign height may not exceed the greater of: (a) 35 feet above frontage street pavement grade; or (b) 20 feet above grade at the base of the sign. Item-3/3 BOARD’S DECISION: BOA MEETING JUNE 13, 2022 POSTPONED TO JULY 11, 2022 (AE DENIAL) 1. The variance is necessary because strict enforcement of the Article prohibits and reasonable opportunity to provide adequate signs on the site, considering the unique features of a site such as its dimensions, landscape, or topography, because: 2. The granting of this variance will not have a substantially adverse impact upon neighboring properties, 3. The granting of this variance will not substantially conflict with the stated purposes of this sign ordinance, because: 4. Granting a variance would not provide the applicant with a special privilege not enjoyed by others similarly situated or potentially similarly situated, because: FINDING: because: OR, OR, AND, ______________________________ Elaine Ramirez Executive Liaison ____________________________ Jessica Cohen Madam Chair Item-3/4 From: To: Subject: Date: Ramirez, Elaine Case Number C16-2022-0004 hearing on 07-11-2022 Friday, July 08, 2022 8:40:15 PM *** External Email - Exercise Caution *** To Board of Adjustment and Elaine Ramirez: My name is Horatiu Pivoda, and I am the property owner of 1101 Bentwood Rd, in Austin. My property is adjacent to 4507 North Interstate 35. This email is in regards to the sign proposal for Apple Tree Holdings; case number C16-2022-0004. I would like the board to know that I am in full support of the proposed sign alterations. And I would like my email to be entered into record that I, as a property owner and a neighbor, have no objections to the proposed changes. In fact, it is my belief that having businesses improve their locations, and visibility will create a positive feedback in the neighborhood. Which we will all benefit from, as a community and as neighbors. If there are further questions, I can be reached via phone @ 714-272-7672 or simply via my email. Thank you Best Regards, Horatio Pivoda | RoCaL Group Inc. c: (714) 272-7672 This email may contain material that is confidential or proprietary. If you are not the intended recipient, please contact the sender and delete all copies. CAUTION: This email was received at the City of Austin, from an EXTERNAL source. Please use caution when clicking links or opening attachments. If you believe this to be a malicious and/or phishing email, please forward this email to cybersecurity@austintexas.gov. Item-3/5 July 8, 2022 Tony Nguyen 4507 N Ih 35 Svrd Nb Austin TX, 78722 Re: C16-2022-0004 Dear Tony, Property Description: LOT 1 TERRY ROSS ADDN Austin Energy (AE) has reviewed your application for the above referenced property, requesting that the Board of Adjustment consider a variance(s) from LDC Sections 25-10-123 (B)(2)(a)(i) and (B)(3)(a) at 4507 N IH 35. After researching this site and our facilities near the existing sign, it has been determined that the sign and its proposed expansion potentially fails to meet required clearances detailed in Austin Energy’s Electric Design Criteria Manual. Please contact electric design lead Brian Van Dyke via email at Brian.VanDyke@austinenergy.com to coordinate this project. Once any clearance conflicts have been resolved, Austin Energy will be able to approve this variance request. Please use this link to be advised of our clearance and safety requirements which are additional conditions of the above review action: https://library.municode.com/tx/austin/codes/utilities_criteria_manual?nodeId=S1AUENDECR_1 .10.0CLSARE If you require further information or have any questions regarding the above comments, please contact our office. Thank you for contacting Austin Energy. Cody Shook, Planner II Austin Energy Public Involvement | Real Estate Services 2500 Montopolis Drive Austin, TX 78741 Cody.Shook@austinenergy.com Item-3/6 Statistical Analysis of the Relationship between On-Premise Digital Signage and Traffic Safety H. Gene Hawkins, Jr., Ph.D., P.E. Associate Professor and Research Engineer Zachry Department of Civil Engineering Texas A&M University Pei-Fen Kuo Graduate Research Assistant Texas A&M Transportation Institute by and Dominique Lord, Ph.D. Associate Professor and Research Engineer Zachry Department of Civil Engineering Texas A&M University Sponsored by Signage Foundation, Inc. P.O. Box 14392 Washington, DC 20044 Texas Engineering Extension Service The Texas A&M University System College Station, TX 77843 December 17, 2012 i Item-3/7 DISCLAIMER The contents of this report reflect the views of the authors, who are responsible for the facts and accuracy of the information presented herein. This document is disseminated under the sponsorship of the U.S. Department of Transportation University Transportation Centers Program in the interest of information exchange. The U.S. Government assumes no liability for the contents or use thereof. The U.S. Government and the State of Texas do not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the object of this report. ACKNOWLEDGMENT The authors wish to recognize the Signage Foundation, Inc., for providing the funding for this research effort. The authors also wish to acknowledge the sign companies that provided proprietary information regarding the installation of the digital signs that were used to create the databases analyzed in this project. Although they are not recognized by name in order to protect the proprietary nature of the information, their contributions are greatly appreciated. ii Item-3/8 ABBREVIATIONS The abbreviations shown below are used in this report. Akaike Information Criterion Bayesian Information Criterion Annual Average Daily Traffic Average Daily Traffic AADT ADT AASHTO American Association of State Highway and Transportation Officials AIC ANOVA Analysis of Variance BIC CEVMS Commercial Electronic Variable Message Signs CG DF EB EBB FHWA HSIS HSM LCD LED MS MSE MST RTM SAR SEM SFI SPF SS SSE SST TTI Control Group Degrees of Freedom Empirical Bayes Electronic Billboard Federal Highway Administration Highway Safety Information System Highway Safety Manual Liquid Crystal Display Light-Emitting Diode Mean of Sum of Squares Error Mean Square Treatment Mean Square Regression to the Mean Spatial Autoregressive Model Spatial Error Model Signage Foundation, Inc. Safety Performance Function Sum of Squares Sum of Squares for Error Total Sum of Squares Texas A&M Transportation Institute iii Item-3/9 TABLE OF CONTENTS Page List of Figures ................................................................................................................................. v List of Tables ................................................................................................................................. vi Executive Summary ...................................................................................................................... vii Chapter 1: Introduction ................................................................................................................... 1 Research Approach ................................................................................................................... 1 Description of a Digital Sign .................................................................................................... 1 Research Activities and Report Organization ........................................................................... 2 Chapter 2: Background Information ............................................................................................... 3 On-Premise Digital Signs ......................................................................................................... 3 Off-Premise Digital Signs ......................................................................................................... 4 Safety Effects ...................................................................................................................... 4 Characteristics of the Evaluation Methods Used in Previous Studies ................................ 7 Chapter 3: Study Data ..................................................................................................................... 9 Crash Data ................................................................................................................................. 9 Sign Data ................................................................................................................................. 11 Data-Merging Procedure ......................................................................................................... 12 Chapter 4: Study Methodology ..................................................................................................... 16 A Before-After Study and a Cross-Sectional Study ............................................................... 16 The Before-After Study .................................................................................................... 16 Common Methods for Conducting a Before-After Study ....................................................... 18 Naïve Method.................................................................................................................... 18 Control Group Method ...................................................................................................... 19 Empirical Bayes Method................................................................................................... 20 Calculation Procedures and Examples .................................................................................... 21 Chapter 5: Results ......................................................................................................................... 25 Individual and Combined Results ........................................................................................... 25 Results for Crashes Related to Multiple and Single Vehicles ................................................ 28 Results for Crashes Related to Different Types of Signs ....................................................... 29 Impact of Sign Hold Time ...................................................................................................... 32 Chapter 6: Summary and Conclusions .......................................................................................... 34 Chapter 7: References ................................................................................................................... 36 Appendix A: Step-By-Step Instructions for Students to Record Sign Data ................................. 39 Appendix B: Statistical Symbols .................................................................................................. 41 iv Item-3/10 LIST OF FIGURES Page Figure 1. Summary of study results ............................................................................................. viii Figure 2. The flow chart for data collection and merging procedure ........................................... 13 Figure 3. Example work table of site data collection .................................................................... 15 Figure 4. A comparison of sample sizes from similar studies ...................................................... 26 Figure 5. The safety effectiveness index and the 95 percent confidence interval for each state (all crash types) ...................................................................................................... 27 Figure 6. The safety effectiveness index and the 95 percent confidence interval for each state (multi-vehicle crashes) .......................................................................................... 28 Figure 7. The safety effectiveness index and the 95 percent confidence interval for each state (single-vehicle crashes) ......................................................................................... 29 Figure 8. The histogram of digital signs for each sign dimension ................................................ 31 Figure 9. Example screenshot of Google Maps ............................................................................ 40 Figure 10. Example screenshot of Google Earth .......................................................................... 40 v Item-3/11 LIST OF TABLES Page Table 1. Safety effects of off-premise digital signs ........................................................................ 6 Table 2. HSIS crash coverage and roadway length by state ......................................................... 10 Table 3. Coefficients for multi and single-vehicle crash regression model .................................. 22 Table 4. Sign site sample size yield .............................................................................................. 26 Table 5. Results of statistical analysis of before-after crash condition ......................................... 27 Table 6. The typical form of a one-way ANOVA table ............................................................... 30 Table 7. Analysis of variance table (color) ................................................................................... 31 Table 8. Analysis of variance table (sign dimension) ................................................................... 32 Table 9. Analysis of variance table (six business types) .............................................................. 32 Table 10. Analysis of variance table (two business types) ........................................................... 32 Table 11. Summary of sign hold times ......................................................................................... 33 Table 12. Example work table of site data collection procedure .................................................. 39 vi Item-3/12 EXECUTIVE SUMMARY The use of digital on-premise signs, which are typically business-related signs that have the ability to change the displayed message, has increased significantly in recent years. On-premise digital signs are located on the same property as the businesses they promote, and some part — or a significant part in some cases — of the sign contains a digital display that can be programmed to change the message at pre-set intervals. Because the use of these signs has increased, jurisdictions have used local sign codes or ordinances to regulate the manner in which digital messages are displayed. Jurisdictions typically justify these regulations by citing traffic safety impacts. However, no comprehensive and scientifically based research efforts have evaluated the relationship between on-premise digital signs and traffic safety. In this study, researchers collected large amounts of sign and crash data in order to conduct a robust statistical analysis of the safety impacts of on-premise digital signs. The statistical tools used the latest safety analysis theory developed for analyzing the impacts of highway safety improvements. The research team acquired the crash data from the Highway Safety Information System, which is a comprehensive database of crash records from several states. One of the advantages of these data is that they also include information about roadway characteristics, such as the number of lanes, speed limit, and other factors. The research team then acquired information about the location of on-premise digital signs from two sign manufacturing companies. Through significant effort by the researchers, these two datasets were merged into a single dataset that represented potential study locations in California, North Carolina, Ohio, and Washington. Of the initial set of over 3,000 possible sites, the research team was able to identify 135 sign locations that could be used for the safety analysis. Potential sites were eliminated from consideration due to any of the following factors: The sign location was not on a roadway that was included in the crash dataset; only major roads were represented in the crash data. The sign location provided by a sign manufacturing company could not be verified through online digital images of the location. Only signs installed in calendar years 2006 or 2007 could be included in order to have adequate amounts of crash data before and after the sign was installed. The research team then used the empirical Bayes method to perform a before-after statistical analysis of the safety impacts of the on-premise digital signs. In a before-after study, the safety impact of a treatment (in this case, the installation of an on-premise digital sign) is defined by the change in crashes between the periods before and after the treatment was installed. However, simply comparing the crash frequencies (known as a naïve before-after analysis) is not adequate to account for factors such as regression to the mean (a statistical concept that explains why after data can be closer to the mean value than the before data) and to provide a means of controlling for external factors that can also cause a difference in crash frequencies. The empirical Bayes method represents the recommended procedure for evaluating the impacts of safety treatments because it overcomes the deficiencies of the naïve method. The safety impacts are represented by the safety index, which is indicated by the symbol . In simple terms, the safety index represents a ratio of safety in the after period compared to safety in the before period, although it is not as vii Item-3/13 simple as dividing the crashes in the after period by the crashes in the before period. A safety index greater than 1.0 indicates an increase in crashes in the after period, and a value less than 1.0 indicates a reduction in crashes in the after period. However, because of the variability in the crash data, the analysis must have statistical validity. Statistical variability is established by defining the 95 percent confidence interval for the safety index, which is based on factors such as sample size and the variability of the data. If the 95 percent confidence interval includes the value of 1.0, then there is a 95 percent chance that there is no statistically significant change in crashes between the before and after periods. The results of the statistical analysis are presented in Figure 1. This figure shows that the safety index for all of the states was 1.0 with a 95 percent confidence interval that ranged from 0.93 to 1.07. This indicates that, for the 135 sites included in the analysis, there was no statistically significant change in crashes due to the installation of on-premise digital signs. The same can also be said about the results for each of the four states on an individual basis because the confidence interval for safety index for each state includes 1.0. The larger confidence intervals for some of the states are due to greater variability in the data and/or smaller sample sizes. The researchers also analyzed single-vehicle and multi-vehicle crashes and found the same result of no statistically significant change in crashes. Finally, the researchers performed an analysis of variance for the sign factors of color, size, and type of business and found no statistically significant differences in the mean safety index values for individual factors. Figure 1. Summary of study results The results of this study provide scientifically based data that indicate that the installation of digital on-premise signs does not lead to a statistically significant increase in crashes on major roads. viii Item-3/14 CHAPTER 1: INTRODUCTION For many generations, most signs — including both traffic and business signs — were static. They displayed only one message that did not change with time. Advances in information display technologies in recent years have led to an increase in the use of many types of digital signs, particularly in the area of on-premise and off-premise business signs. On-premise digital signs provide the ability to communicate a wide variety of messages and to change the manner in which the message is presented over time. As such, these digital signs represent a significant advancement in communication technologies and the ability to deliver valuable marketing information to potential customers. However, some groups have raised questions related to the traffic safety aspects of business signs that change messages on a frequent basis. The traffic safety concerns are often related to issues of potential driver distraction from the roadway due to the dynamic nature of these signs. These safety concerns are sometimes addressed through local regulation of these types of signs, which may prohibit or limit the use of on-premise digital signs. These regulations tend to be developed at the local level and do not have a significant level of scientific, nationally based research supporting the regulations. The traffic safety concerns associated with on-premise digital signs have existed for some time, but there has been little research, particularly on a national level, that directly addresses the safety impacts of on-premise digital signs. In part, this is due to the fact that the use of such signs has grown only in the last 5–10 years. The research described in this report was conducted to provide a scientifically based, national analysis of on-premise digital signs so that the traffic safety impacts of such signs can be better understood. RESEARCH APPROACH The basic research method used in this study is a before-after statistical analysis of the change in traffic crashes at locations where digital signs were installed. The research team used digital sign installation information provided by sign manufacturers to identify locations in selected states where digital signs had been installed in the 2006–2007 time frame (this time frame was selected to provide adequate numbers of crashes in both the before and after periods). The analysis locations were limited to California, North Carolina, Ohio, and Washington because these states are part of the Federal Highway Administration (FHWA) Highway Safety Information System (HSIS). The HSIS is a database of crash records that includes detailed information about the roadway and crashes, including such factors as the number of lanes, the speed limit, crash severity, and other factors. The researchers then mapped the sign sites to the crash datasets to identify locations with crashes. These locations were then analyzed to compare the crashes before installation of the digital sign to the crashes after installation of the sign using statistical analysis procedures. DESCRIPTION OF A DIGITAL SIGN For the purposes of this study, a digital sign is defined as a sign that uses an electrical display, such as a liquid crystal display (LCD) or light-emitting diode (LED), to provide changeable 1 Item-3/15 messages or graphics. There are several types of digital signs, including digital billboards, indoor video advertisements, and street-level advertisements (such as LED signs on bus shelters). For this study, the researchers focused only on on-premise digital signs, which are signs located on the same property as the business with which they are associated. The research effort did not include or address off-premise signs or billboards. RESEARCH ACTIVITIES AND REPORT ORGANIZATION There were five major activities associated with this research effort. The study began by reviewing and evaluating previous research on the safety aspects of digital signs and the statistical methods that other researchers have used to evaluate the safety aspects of signs. Chapter 2 describes the results of the review of background information. The researchers then began to collect information related to digital signs and crash data in the selected states. The sign information included the location and date of installation, and the crash data included the location and date. The researchers then devoted extensive effort to matching the locations and dates of the signs and crash datasets. Chapter 3 describes the sign and crash data and how the two datasets were merged together. Once this was accomplished, the next step was to develop a valid and scientifically based statistical analysis procedure to determine if there were any statistically significant changes in crashes after installation of digital signs. Chapter 4 describes the development of a statistical methodology, including a comparison of the advantages of the different options for conducting the statistical analysis. Finally, the research team used the results of the statistical analysis to define the key study findings, which are described in Chapter 5. Chapter 6 presents the conclusions and recommendations for the research study. 2 Item-3/16 CHAPTER 2: BACKGROUND INFORMATION This chapter provides a review of the literature related to on-premise digital signs and their impacts on traffic safety. The review also includes a summary of statistical methods that can be used for evaluating the safety effects for these types of signs. Although the majority of the work has been related to off-premise digital signs, key studies associated with off-premise signs are nonetheless briefly discussed here. It should be pointed out that compared to other types of roadway-related operational and design features, such as access point density on urban arterials or on-street parking designs, the number of documents that are related to either on- or off- premise signs is relatively small. On-premise signs are signs that are located on the same property as the activity described in the sign, while off-premise signs are located away from the activity identified in the sign. Off- premise signs are also known as third-party signs or outdoor advertising, and the most common example is a billboard. In general, off-premise signs have a larger visible area, which is attributed to the fact that these signs usually have greater surface areas and have higher mounting heights than on-premise signs. Furthermore, off-premise signs have a larger viewership because they are usually located adjacent to freeways and major highways with higher traffic volume. On the other hand, on-premise signs are installed on private property where a company conducts its business, and most are located along urban streets or local roadways. According to The Signage Sourcebook (U.S. Small Business Administration, 2003), the viewing opportunities for outdoor advertising (typically 333,350 cars per day) are much greater than those for an on-premise sign (30,000 cars per day). The literature review is divided into two sections. The first section summarizes studies related to on-premise digital signs. The second section presents the summary of two key studies associated with off-premise digital signs. ON-PREMISE DIGITAL SIGNS This section describes the characteristics of the studies that have examined the relationship between safety and on-premise digital signs. To the knowledge of the authors, only two studies have investigated this relationship. It should be pointed out that the safety relationships identified in these research documents were not based on crash data but more on opinions and hypotheses, which limits their value as a direct measure of on-premise sign safety. The first study was conducted by Mace (2001). This author performed a literature review and listed two hypotheses about how on-premise signs can influence crash risk. The first hypothesis states that on-premise business signs distract drivers’ attention from their primary driving tasks, resulting in higher crash risks. The second hypothesis asserts that on-premise business signs may mask the visibility of regulatory and warning road signs, which also can negatively influence crash risk. On the other hand, Mace (2001) noted positive effects associated with commercial signs. He reported that commercial signs could reduce unnecessary traffic exposure by providing adequate navigation information for drivers, such as providing restaurant information for hungry drivers. 3 Item-3/17 However, only measuring the frequency and duration of drivers’ distraction may not represent the safety impacts of on-premise signs because a study published earlier showed that half of the objects that drivers see are not related to driving tasks (Hughes and Cole, 1986). In other words, besides on-premise signs, other roadside features may also distract drivers. The possible solution to minimize the negative effects of an on-premise sign, but still keep its positive effects, is to separate the sign’s content to primary (navigation) and secondary (commercial) information. Although, in the past, on-premise signs and off-premise signs were treated as distinct signage, they are becoming more homogeneous in terms of characteristics. In the second study, Wachtel (2009) mentioned that more roadside businesses, especially those with multiple users (e.g., shopping centers, auto malls, sports complexes, and entertainment places), now install larger- sized on-premise digital signs because of the lower cost and better performance of the LED display. Wachtel indicated that the largest digital advertising sign in the world is an on-premise sign in New York City. This sign is 90 ft tall and 65 ft wide, and is mounted on a 165-ft-tall steel post on the roof of the warehouse. The visible distance is over 2 miles. Wachtel also suggested that some on-premise signs affect traffic safety more than some off-premise digital signs because the locations and elevations of on-premise signs might be closer to the road users. In addition, the angles of on-premise signs may be out of the cone of vision and require extreme head movements to read. In summary, these two studies showed more research is needed for understanding the relationship between on-premise digital signs and crash risk. OFF-PREMISE DIGITAL SIGNS This section is divided into two parts. The first part describes two key studies that have examined the safety effects of off-premise digital signs. The second part covers methodologies that have been used for estimating these effects. Safety Effects There are two reports that provide reviews of the findings, methods, and key factors related to the safety effects of off-premise digital signs. The first systematic study related to the impacts of off-premise signs was conducted 11 years ago by Farbry et al. (2001). Their study reviewed earlier reports and analyses (including those about electronic billboards and tri-vision signs) and provided the foundation for the second study written by Molino et al. (2009). In the second report, Molino et al. (2009) reviewed 32 related studies, which included those initially reviewed by Farbry et al. (2001), and noted that the majority of studies reported a negative effect between digital billboards and traffic safety. Although the number of studies that showed harmful impacts is five times more than the number of studies that showed no harmful impacts, the authors suggested that this ratio may not be strong evidence to prove the negative effects linked to electronic billboards (EBBs). The individual studies considered by these researchers had very different study methods and statistical powers, which can have a significant effect on the quality and results of the research. 4 Item-3/18 Another important finding in the Molino et al. (2009) report is that drivers usually have spare attention capacities, and they can be distracted from their driving tasks by roadside objects (such as EBBs). However, these distractions may be riskier when the driving demands increase, such as in fixed hazard areas (e.g., intersections, interchanges, and sharp curves), in transient risky conditions (e.g., adverse weather, vehicle path intrusions, and slow traffic), or when other important information is processed at the same time (e.g., an official traffic sign). In other words, not only will the sign’s internal characteristics (overall size, legend size, color, contrast, luminance level, etc.) affect crash risk, but so will external environmental factors (type of road, speed, weather conditions, time of day, etc.). Hence, Molino et al. list all possible key factors and suggest further studies to examine how they could influence safety. These factors are categorized into two groups: independent and dependent variables. The independent variables are separated by subject into five subgroups: billboard, roadway, vehicle, driver, and environment. It should be noted that the relationship between EBBs and on-premise signs is discussed in the environment subgroup, and dynamic factors of on-premise signs, such as change rate, motion, video, and sound, are listed as extremely important. The dependent variables are separated into vehicle behavior, driver/vehicle interaction, driver attention/distraction, and crash categories. Since there are hundreds of related key factors, the authors claimed that “No single experiment can provide the solution” and suggested future research programs to address the following topics: (1) determining when distraction caused by commercial electronic variable message signs (CEVMSs) affects safe driving, (2) investigating the relationship between distraction and various CVEMS parameters, and (3) examining the relationship between distraction and safety surrogate measures, such as eye glance and traffic conflicts. Table 1 summarizes the literature review results from these two reports. This table shows that the results of crash studies are not consistent, and most studies have some important weaknesses, such as neglecting biases related to the regression to the mean (RTM) (discussed below) and site- selection effects (using the naïve method), low statistical power, and analysis results based on erroneous assumptions. It should be noted that only post-hoc crash studies are listed here because this study focuses on the change of crash rate caused by on-premise digital signs. As mentioned, Table 1 shows that the results related to the safety effects of off-premise signs are inconsistent. The inconsistencies can be fully or partly attributed to various study limitations. For instance, the studies in the Wachtel and Netherton report (1980) and Wisconsin Department of Transportation report (1994) both used a naïve before-after study methodology (methodology approaches are described in Chapter 4), and they did not account for the RTM bias, which may change their estimates of crash rate and safety effects of signs. The general idea of RTM is that when observations are characterized by very high (or low) values in a given time period and for a specific site (or several sites), it is anticipated that observations occurring in a subsequent time period are more likely to regress toward the long-term mean of a site (Hauer and Persaud, 1983). Also, these studies should provide the variance of estimators (that is the uncertainty associated with the estimator) for judging the statistical significance of their results. Moreover, grouping studies where the objectives or types of signs are different is not appropriate. For example, the goal of the report prepared by Tantala and Tantala (2007) was to study the safety impacts caused by converting traditional billboards to digital billboards, while other studies focused on the safety impacts after installation of new digital billboards. Those are two distinct effects that are examined and should not be grouped together to evaluate the safety effects of on-premise digital 5 Item-3/19 signs. Wachtel (2009) also noted other limitations in Tantala and Tantala’s study, such as a lack of adequate before-after and comparison group data; no clear definition and reasonable calculation of the visual range and legibility range of EBBs; and no crash data related to adverse weather, impaired drivers, and interchanges. Table 1. Safety effects of off-premise digital signs Study Methods Data Type Results Location Sample Size Wachtel and Netherton (1980) Naïve before- after study Crash frequency The crash reduction of target area was 10% less than the overall reduction (after the installation of the signs) Tele-Spot sign, Boston Not provided Wisconsin Department of Transportation (1994) Naïve before- after study Crash frequency, Average daily traffic (ADT) Smiley et al. (2005) Before-after study (empirical Bayes) Crash frequency, ADT, safety performance function Before-after study (control group) Crash frequency, ADT, control group Crash rate (eastbound): all crashes increased 36%, sideswipe crashes increased 8%, and rear-end crashes increased 21% Crash rate (westbound): all crashes increased 21%, sideswipe crashes increased 35%, and rear-end crashes increased 35% Downtown intersection sites: no significant change in crash rate (all crashes increased 0.6%, injury crashes increased 43%, and rear-end crashes increased 13%) Rural sites: no significant change in crash rate based on most compared sites Tantala and Tantala (2007) Tantala and Tantala (2009) Naïve before- after study Crash frequency, control group, ADT No description of the method No significant change in crash rate No significant change in crash rate Milwaukee, Wisconsin 2 Toronto, Canada Toronto, Canada Cuyahoga, Ohio Cuyahoga, Ohio 3 1 7 7 The second shortcoming in Tantala and Tantala (2007) is that they used a simple correlation analysis between sign density and crash rate to examine safety effects of billboards. Using this approach, they found that the correlation coefficients among the scenarios analyzed were very low (around 0.20), indicating that the installation of billboards did not increase the number of crashes. This may well be true, but they did not use the right analysis tool. For investigating the relationship between sign density and the number of crashes, it is more appropriate to develop one or several regression models since the safety analyst can have a better control over other factors that can influence the number and severity of crashes (Lord and Mannering, 2010). In a regression model, several independent variables can be included, which is better to estimate the variable of interest (such as the installation of digital signs). However, it should be pointed out that the before-after study, as performed in this study, still remains the best methodological approach for estimating the safety effects of an intervention. Among all studies in Table 1, Smiley et al. (2005) provides the more reliable results since they used a before-after method using a control group (CG) and empirical Bayes (EB) approach. The 6 Item-3/20 only limitation is related to the small sample size. The authors of the study only evaluated three sites. Even with a small sample size, the EB method can still be successfully used to evaluate the safety effects of an intervention, as was done by Ye et al. (2011). Ye et al. (2011) used the EB method to estimate the safety impacts of gateway monument signs, which can be categorized as one type of off-premise sign. Gateway monuments are roadside structures used to introduce a city or town. These monuments usually have the name of the city or town and are located at the city limits. According to Wachtel et al. (2009) and Farbry, (2001), using crash data might not be a precise method because crashes usually have multiple causal events, which are difficult to extract from crash datasets. For example, they noted that sign internal variables (such as size, brightness, viewing angle, etc.) might play main roles in drivers’ distraction or ignoring of official traffic signs, while other external factors affect conflicts and crash risk. Although those reasons may be legitimate, utilizing crash data is still the best approach for evaluating the safety effects of interventions as well as those associated with operational and design features (Hauer 1997). As stated by Hauer, “It follows that, in the final account, to preserve the ordinary meaning of words, the concept of safety must be linked to accidents.” Furthermore, using crash data have other advantages: lower cost and fewer artificial errors. Firstly, the cost of conducting a before-after crash study is much lower than human-centered methods because the researchers do not need to purchase equipment and hire participants for conducting driving tests. Secondly, crash data are based on crash reports, which can provide a more accurate measure of safety than surrogate measures such as speed, driver behavior, or other measures. Only by conducting a before-after crash study can one provide results that combine multiple casual variables in the real world. Other methods cannot displace the above advantages, which explain why the research team selected the before-after methodology for estimating the safety effects of digital signs. Characteristics of the Evaluation Methods Used in Previous Studies This section describes the characteristics of other methods used in previous studies for examining the safety effects of off-premise digital signs. In addition to a crash before-after study approach, the most common study methods that have been used for examining the safety impacts of off-premise signs include eye fixations, traffic conflicts, headways and speeds, and public surveys. Most studies used one or more of the above methods to examine the impacts of off- premise signs (Molino et al., 2009). For instance, Smiley et al. (2005) used four different methods (eye fixation, conflict study, before-after crash study, and public survey) for examining a video sign located in Toronto. On the other hand, Lee et al. (2007) used eye fixations and a questionnaire for their study. It should be noted that the results from multiple measurements are usually inconsistent. Briefly, the eye fixation study method uses an eye-tracking system to record drivers’ eye movements. The results (e.g., eye glances and durations) can provide direct evidence of where drivers are looking while driving, leading to assumptions as to whether drivers are distracted when they are driving near or toward a sign (or at other roadside features). Traffic conflicts, often referred to as surrogate measures of safety, can be used for identifying risky driving behaviors, such as braking without good reason, inappropriate lateral lane displacement, and delays at the start of the green traffic signal phase. Headways and vehicle speed can be used to 7 Item-3/21 assess distracted drivers since those drivers tend to have shorter headways and higher speed variances. Most details about experiment design, such as the participant number, study site size, driving route length, and experiment duration can be found in Appendix B of the report prepared by Molino et al. (2009). In the current study, the researchers focus the discussion on the before-after crash data study method for two reasons. First, Molino et al. (2009) did not provide a detailed experimental design for using crash data, and some studies were criticized for inappropriate methodology (Tantala and Tantala, 2007; 2009). Second, the costs associated with other experimental methods are significant and are greater than the resources that were allocated for the current research study. According to Molino et al. (2009), the budgetary costs to conduct research using other experimental methods vary between $0.4 million and $0.8 million for using on-road instrumented vehicles, $2 million and $4 million for conducting a naturalistic driving study, and $1 million and $3 million for using an unobtrusive observation approach. 8 Item-3/22 CHAPTER 3: STUDY DATA To conduct the safety analysis, the research team had to develop plans for collecting the necessary data, manipulating the data into a format that could be used for the safety analyses, and then conducting the statistical analysis to identify the safety impacts of on-premise digital signs. The success of this project relied upon the ability to acquire two distinct sets of data and the robustness of the individual datasets. The two datasets needed for the analysis included (1) information regarding the location and installation dates for on-premise digital signs, and (2) data regarding crash histories on the roadways in the vicinity of the on-premise digital signs. The latter also included information about operational (e.g., traffic flow and speed limit) and geometric (e.g., functional class and lane width) design features located at and adjacent to the on- premise digital signs. From the beginning of the project, the research team expected to use the HSIS crash data for the crash history dataset. The real challenge of this project was identifying specific information about on-premise digital signs for the states represented in the HSIS, and the researchers encountered numerous challenges in acquiring this information. Once the data for both groups were acquired, the researchers had to overcome differences in the datasets so that the data could be merged into a single dataset for analysis. The activities associated with the acquisition of the crash data, acquisition of the sign data, and the merging of the two datasets are described in this chapter. CRASH DATA The HSIS is operated and maintained by the FHWA, and is widely used for safety research programs that provide input for public policy decisions. The HSIS is a multistate relational database that contains crash, roadway, and vehicle information. Crash information/files contain basic crash information, such as location (based on reference location or mile-point), time of day, lighting condition (e.g., daylight, dark and no lighting, dark and roadway lighting, etc.), weather conditions, crash severity, the number of related vehicles, and the type of crash (e.g., head-on, right angle, sideswipe, etc.). Each row in the spreadsheet file contains crash information for individual crashes and a unique ID number, and each column represents a variable. The roadway information/files provide traffic and geographic information for each roadway segment, such as annual average daily traffic (AADT), speed limit, beginning mile-point, end mile-point, number of lanes, lane and median width, shoulder width and type, rural or urban designation, and functional classification. The vehicle information/files contain driver and vehicle information, such as a crash identification number, driver gender, driver age, contributing factor (possible casual factor), vehicle type, and others. These individual file types can be linked together as a whole dataset. For example, crash files and road files can be linked by their location information (route number and mileage), or crash files and vehicle files can be linked together by their crash identification number. Currently, there are seven states that actively participate in the HSIS: California, Illinois, Maine, Minnesota, North Carolina, Ohio, and Washington. However, the HSIS has an upper limit on the amount of data that can be requested by researchers (including the number of states, the request area, and total variables). To maximize the value of the crash data that they could request, the 9 Item-3/23 research team held discussions with the research advisory panel to identify the states (from the list of seven HSIS participating states) where there would be higher concentrations of on-premise digital signs. Based on this input, the research team requested HSIS data for California, North Carolina, Ohio, and Washington in order to get a maximum number of study sites. All crash datasets were downloaded from the HSIS website and stored in a spreadsheet format. The definitions for the variables in a state’s crash data were found in the HSIS guidebooks. It should be noted that each state has its own guidebook and data record format. In other words, one specific variable might be available for some states, but this variable may have different meanings or category types, or even be unavailable for other states. The inconsistent definitions among different states’ crash datasets can affect the quality of analysis and results when selecting specific variables for identifying target crashes (such as rear-end crash) needed for more advanced analysis. The differences between states also create challenges when trying to merge data into a single dataset for analysis. Although the HSIS dataset provides the most comprehensive crash data from different states, the HSIS has some limitations. First, the HSIS only includes crashes that occur on major roads, such as interstate highways, U.S. highways, and state highways. The HSIS dataset may not include crash-related data for secondary roads in rural areas or city streets in urban areas, including arterial streets that are major roads in a city but are not on the state highway system. Table 2 identifies the level of crash coverage and roadway length for each state selected for the analysis. California North Carolina Table 2. HSIS crash coverage and roadway length by state 1. More than 500,000 crashes occur each year; HSIS includes about 38% of those crashes. 2. HSIS includes 15,500 miles of mainline (non-ramp) roadways. 1. About 230,000 crashes occur each year; HSIS includes 70% of those crashes. 2. Of the 77,000 miles of roadway on the North Carolina state system, approximately 62,000 miles are included in the database. Ohio 1. About 380,000 crashes occur each year; HSIS includes 40% of those crashes. 2. In Ohio, about 116,000 miles of highway in total; HSIS includes approximately 19,500 miles of roadway. Washington 1. 130,000 crashes occur each year; HSIS includes 37% of those crashes. 2. HSIS contains 7,000 miles of mainline (non-ramp) roadway. Another limitation of the HSIS data is that the dataset is not continuously updated. The HSIS data represent the final crash datasets from each state after the state has processed the crash data. As a result, the HSIS dataset may not include the last several months or more of crash data from a state. Currently, the most updated HSIS crash data are through 2009 (California is updated to 2008), so the most recent one or two years of crashes are not included in the HSIS data. Also, the oldest HSIS crash data extend back only through 2004. Limiting crash data to the period from 2004 to 2009 was a significant consideration in this research project because the large growth of on-premise digital signs is relatively recent, having mostly grown since the mid- to late 2000s. The lack of data for the last two to three years created challenges with respect to developing a robust statistical analysis procedure. For a comparison of safety impacts of a treatment (such as installation of a digital sign) to be meaningful, both the before and after analysis periods need to be about equal and as long as possible. This meant that, to have two-year analysis periods (two years before and two years after) in the safety analysis, on-premise digital signs needed to be 10 Item-3/24 installed in either 2006 or 2007. In order to focus the safety analysis on the long-term impacts of on-premise digital signs, the researchers did not include the calendar year of installation of a sign in the analysis. For example, if a sign was installed in 2006, the before period was calendar years 2004 and 2005, and the after period was calendar years 2007 and 2008. An additional limitation of the HSIS crash data is that the crash location within the HSIS is identified to the nearest 0.1 mile (528 ft) on the roadway. This required the safety analysis to be conducted for the tenth of a mile length of roadway that a sign was located within. The level of accuracy is the primary reason that 0.1 miles was chosen as the effective area of the sign. The researchers viewed the limitations mentioned above as minor and ones that had minimal impact on the study results. There are no comparable crash datasets available to researchers that could be used for a similar type of analysis of crashes. The only alternative available to the researchers would have been to try and obtain crash data from individual agencies where on- premise digital signs have been installed. Such an approach may have provided more specific data about individual signs and site characteristics, but would have resulted in an extremely small dataset. The researchers felt that such small sample sizes would not provide sufficient robustness for statistical analysis and that the approach using the HSIS data provided greater scientific validity and robustness, as discussed in the previous chapter. SIGN DATA With the acquisition of the HSIS data, the research team had information to analyze crashes but had no idea about where to conduct the analysis. Determining the location for the crash analysis required information regarding the location of on-premise digital signs. Furthermore, due to the date limitations of the HSIS data, only sign sites where the sign was installed in 2006 or 2007 could be used for the crash analysis. So the research team began the process of identifying locations in California, North Carolina, Ohio, and Washington where on-premise digital signs had been installed on major roads in 2006 or 2007. Initial attempts to identify sign locations focused upon getting information from the Signage Foundation, Inc., (SFI) research advisory panel. However, the results did not provide a large enough sample size for a robust statistical analysis. The research team began to contact sign installation companies but encountered challenges in acquiring the large amount of data needed to conduct the research. The primary challenge associated with contacting sign installation companies (which are the same companies that market the signs to individual businesses) was the proprietary nature of the business information the research team was requesting. Another challenge was the large number of individual companies that needed to be contacted to develop a robust sample size. Because of the challenges of working with sign installation companies, the research team shifted the focus to sign-manufacturing companies. Eventually, the research team was able to work with two electronic sign-manufacturing companies to get a list of on-premise digital signs installed in any of the four study states during 2006 or 2007. Each of the two lists was converted into datasets for use in the research effort. The first dataset (dataset #1) contained 2,953 sign sites and 27 variables, which included the characteristics of signs and roads, such as sign order date, sign 11 Item-3/25 address (road, county, and state), the nearest cross street and its distance from the sign, the nearby cross street with the highest volume and its distance from the subject intersection, and traffic volume on the subject road. The research team did not use the road information from dataset #1, relying instead upon the road data in the HSIS crash dataset. This ensured consistency in the approach with the different sign datasets. Also, the sign installation date was considered to be the sign order date plus two weeks. This assumption was based on input from the sign- manufacturing company. Since the entire year that the sign was installed was excluded from the analysis, this was considered not to be a critical issue. The second dataset (dataset #2) had 63 site addresses and 10 variables. Unlike the first dataset, most variables in dataset #2 were related to product information, such as installation data, sales representative, product name, matrix, color, customer ID (address), and status of signs. For the analysis, these two datasets were combined as one for use in analyzing the crashes by individual state. The combined dataset was further refined by removing all sign locations that were not installed in either 2006 or 2007. The calendar year that a sign was installed was treated as the construction year, and the crashes that occurred in that year were removed from the analysis. The entire calendar year was removed from the analysis due to uncertainty over the actual installation date of the sign since the data provided only the order date for the sign. Removing the entire calendar year associated with installation also eliminated the novelty effect associated with implementing a new feature. The second variable, the sign installation address, was used to select related crashes by the sign’s location and default sign-effective areas. For example, the researchers defined the crashes located within 0.1 miles from the target signs as related crashes. In reality, the effective area could be larger or smaller depending upon the sign size. The procedure used for this analysis did not adjust the effective area based on sign size or other factors. Overall, significant effort was put into ensuring the accuracy of the sign datasets because the quality of the data had a huge impact on the precision and accuracy of the analysis. DATA-MERGING PROCEDURE The previous sections explain how the researchers obtained their study data (the sign dataset and the crash dataset) and the characteristics of each dataset. This section gives more details about the dataset-merging procedure. Several steps were involved in merging the crash and sign location datasets into a single dataset that could be used for statistical analysis. The early steps focused on confirming that the digital sign was still in place and near the road that it is related to. This was needed because a site could have an address on one road but have the sign facing traffic on another road bordering the site property. The later steps focused upon converting the street address of the sign location to a route and milepost value that could be used with the crash dataset. This complex effort was necessary due to the fact that the sign and crash datasets used different location methods. The sign dataset was based on the site address, while the crash database was based on route number and milepost. For example, a location in the sign dataset would record a location with “1234 North Highway 101, Anytown, WA 98584,” but the HSIS would show the same location as “route number = 23101” and “mile post = 335.72.” In order to define the related crashes that were adjusted to the target signs, the researchers needed to transfer sign locations into the HSIS location system. The basic steps are described below and illustrated in Figure 2. 12 Item-3/26 Figure 2. The flow chart for data collection and merging procedure 1. For each record of the combined sign dataset (3,016 total records), the research team evaluated the location information (typically a street address) and the sign order date. Records with missing or incomplete location information or with assumed sign installation dates that were not in 2006 or 2007 were deleted from the dataset. 13 Item-3/27 2. Research team members then verified the location of the sign using the site address in the sign dataset and taking the steps listed below. Figure 3 shows an example table that the researchers used for the above data collection, including screenshots of Google Maps and Google Earth (Google Earth, 2008). Columns 1–3 are the address information given by the sign companies. Columns 4–7 are determined through Google Maps, and Columns 8–11 are determined through Google Earth. a. The sign was located in Google Maps using the site address. b. Using the Street View feature of Google Maps, a member of the research team identified the sign on the site or deleted the record with a note that the on-premise digital sign could not be identified. There were some challenges associated with finding digital signs using the Street View pictures from Google Maps, including fuzzy pictures with low resolution, which made it difficult to evaluate some signs, and digital signs that were not obvious during the daytime (Street View provides only daytime pictures). c. The screen image of the subject sign was saved, and basic sign characteristics were identified and/or estimated. Examples include sign color, size, and business type. d. An initial determination was made as to whether the sign was located on a major road that would be part of the HSIS crash dataset. If the road was not expected to be a major road, the record was deleted from the dataset. 3. The sign location was entered into Google Earth to determine the county in which the sign was located and the mileage from the county border. This included identifying the county identification code in the appropriate HSIS manual for a given state. This provided the milepost location information needed to relate the sign location to the location information in the crash dataset. Defining the milepost information required doing the following: a. Identifying the neighboring county, which was used to determine in which direction the mileposts were increasing. b. If the county had mileposts restarting at zero at the county borders, determining in which direction they were increasing, based on the number of lanes at the borders. If the direction could not be determined, a general rule of increasing from west to east or south to north was used. c. Using the path tool in Google Earth to measure the distance from the county border to the sign. This distance and the beginning milepost at the county border established the milepost of the sign. An example (using the above procedure) can be founded in Appendix A. After target sign locations were transferred into the HSIS locating system, a statistics software package, “R,” was used to select the related crashes among the whole HSIS dataset. 14 Item-3/28 Figure 3. Example work table of site data collection 15 Item-3/29 CHAPTER 4: STUDY METHODOLOGY Evaluating the effects of treatment on the number and severity of crashes is a very important topic in highway safety. For the last 30 years, various methods have been proposed for evaluating safety treatments (Abbess et al., 1981; Danielsson, 1986; Davis, 2000; Hauer, 1980a; Hauer, 1980b; Hauer et al., 1983; Maher and Mountain, 2009; Miranda-Moreno, 2006; Wright et al., 1988). The methods are classified under two categories: the before-after study and the cross- sectional study. In a before-after study, the safety impacts of an improvement or treatment at a given location are determined by comparing the change in crashes before and after the improvement/treatment was installed. In a cross-sectional study, crashes or crash rates on two different facilities with similar characteristics except for the improvement of interest are compared. The before-after study is typically more desirable because it provides a more direct evaluation of the safety impacts. Although they have been used by some researchers (Noland, 2003; Tarko et al., 1998), cross-sectional studies are more difficult to conduct because different facilities are rarely identical in all features except the one of interest. Hence, the cross-sectional approach was not used in this research. The before-after type of study can be further divided into several types: naïve before-after study, before-after study with control group, before-after study using the EB method, and before-after study using the full Bayes approach. The before-after study using the full Bayes approach is a more recent development in statistical safety analysis, developed and used by several noted safety researchers (Hauer and Persaud, 1983; Hauer et al., 1983; Hauer, 1997; Li et al., 2008; Persaud and Lyon, 2007). The advantages and disadvantages for each of the above before-after methods are described in more detail in this chapter. A BEFORE-AFTER STUDY AND A CROSS-SECTIONAL STUDY As mentioned previously, observational crash studies can be grouped into two types: the before- after study and the cross-sectional study. The selection of the study type is based on the availability of historical crash data, traffic volume, or the comparison group. The following sections provide details about the before-after methodology. The Before-After Study The before-after study is a commonly used method for measuring the safety effects of a single treatment or a combination of treatments in highway safety (Hauer, 1997). Short of a controlled and full randomized study design, this type of study is deemed superior to cross-sectional studies since many attributes linked to the converted sites where the treatment (or change) was implemented remain unchanged. Although not perfect, the before-after study approach offers a 16 Item-3/30 better control for estimating the effects of a treatment. In fact, as the name suggests, it implies that a change actually occurred between the “before” and “after” conditions (Hauer, 2005). As described by Hauer (1997), the traditional before-after study can be accomplished using two tasks. The first task consists of predicting the expected number of target crashes for a specific entity (i.e., intersection, segment where an on-premise sign was installed, etc.) or series of entities in the after period, had the safety treatment not been implemented. In other words, the before-after approach described by Hauer compares the expected number of crashes in the after period with the treatment installed to the expected number of crashes in the after period had the treatment not been installed. The calculation for each expected number of crashes is based on numerous factors, including the actual number of crashes in the before condition, the actual number of crashes in the after period, and incorporation of site-specific and statistical considerations. The symbol is used to represent the expected number of crashes in the after period (a summary of all statistical symbols used in this report are presented in Appendix B). The second task consists of estimating the number of target crashes (represented by the symbol ) for the specific entity in the after period. The estimates of and are ˆ and ˆ (the caret or hat represents the estimate of an unknown value). Here, the term “after” means the time period after the implementation of a treatment; correspondingly, the term “before” refers to the time before the implementation of this treatment (an on-premise digital sign in this study). In most practical cases, either ˆor ˆ can be applied to a composite series of locations (the sum of i’s below) where a similar treatment was implemented at each location. Hauer (1997) proposed a four-step process for estimating the safety effects of a treatment. The process is described as follows (see also Ye and Lord, 2009): Step 1: For i 1, 2, ..., n , estimate ( )i and estimated and predicted values for each site i, such that ˆ Step 2: For i 1, 2, ..., n , estimate the variance for each, ( )i . Then, compute the summation of the ˆ( )i and ˆ ˆ{ ( )} i and Var . ˆ( )i i . For ˆ{ ( )} Var each single location, it is assumed that observed data (e.g., annual crash counts over a long time frame) are Poisson distributed and ˆ( )i can be approximated by the observed i will depend on value in the before period. On the other hand, the calculation of ˆ{ ( )} the statistical methods adopted for the study (e.g., observed data in naïve studies, method of moments, regression models, or EB technique). Assuming that crash data in the before and after periods are mutually independent, then ˆ i { ( )} and ˆ { } Var Var Var Var ˆ { } Var ˆ i { ( )} . Step 3: Estimate the parameters and , where ˆ ˆ ˆ (again, referring to estimated values) is defined as the reduction (or increase) in the number of target crashes between ˆ ˆ/ˆ the predicted and estimated values, and is the ratio between these two values. When θ is less than one, the treatment results in an improvement in traffic safety, and when it is larger than one, the treatment has a negative effect on traffic safety. The term has also been referred to in the literature as the index of effectiveness (Persaud et al., 2001). Hauer (1997) suggests that when less than 500 crashes are used in the before-after study, should be corrected to remove the bias caused by the small sample size using 17 Item-3/31 2Var ]ˆ/}ˆ{ . The total number of crashes was the following adjustment factor: over 500, but the adjustment factor had to be applied when subsets of the data, such as single- or multi-vehicle crashes, were analyzed. 1/[1 and }ˆ{Var . These two variances are calculated }ˆ{Var is also adjusted for the small sample size): Step 4: Estimate the variances }ˆ{Var using the following equations (note: }ˆ{ }ˆ{ Var Var }ˆ{ Var ˆ 2 [( Var }ˆ{ Var )ˆ/}ˆ{ 2 (1[ Var Var ( 2 ˆ/}ˆ{ 2 ˆ/}ˆ{ 2 )] )] (Eq. 1) (Eq. 2) The four-step process provides a simple way for conducting before-after studies. Three common before-after methods will be introduced in the following sections. All three methods use the same four-step process. COMMON METHODS FOR CONDUCTING A BEFORE-AFTER STUDY Having selected the before-after study approach, the research team then needed to decide which specific before-after method would be the most appropriate for analyzing the safety impacts of on-premise digital signs. This section of the report describes the methodologies and data needs associated with three before-after study types: naïve before-after studies, before-after studies with a CG, and the EB method. Naïve Method Among all the before-after methods, the naïve method is the simplest. The estimation of θ is simply equal to the ratio between the number of crashes in the after period and the number of crashes in the before period (which is used to predict the number of crashes in the after period if the treatment was not implemented). Equation 3 illustrates how the index of safety effectiveness is calculated. This method is very straightforward, but it is seldom used in the current safety study because it does not account for the RTM bias. Not including the RTM bias could overestimate the effects of the treatment or underestimate the safety impacts. The naïve method does not account for external factors that occur at the local or regional level, such as changes in weather patterns or economic conditions. ˆ naive ˆ ˆ t j t n i 1 n 1 i N 1 T ij 2 N 1 j T ij 1 (Eq. 3) Where ˆ naive = the estimate of safety effectiveness by using the naïve method, ˆ = the predicted number of crashes for the treatment group in the after period, ˆ = the estimated number of crashes for the treatment group in the after period, n = the sample size, t = the time period, 18 Item-3/32 T 1 2 T ijN = the observed response for site i (T = treatment group) and year j (in the before period), and = the observed response for site i (T = treatment group) and year j (in the after period). ijN The result can be adjusted when the traffic flow and time interval are different between the before and after periods. It is adjusted by modifying the predicted number of crashes as shown in Equation 4: N (Eq. 4) n t r r d f 1 i j 1 T 1 ij = the ratio of the duration between the after and before periods, and = the ratio of the traffic flow between the after and before periods. Where dr fr Control Group Method The CG method can be used to help control for external factors. The number of crashes collected at the control sites is defined as µ (before) and ν (after). The adjusting factor, the ratio of ν to µ, is used to remove the effects caused by other external factors from π in the theorem. Equation 5 illustrates how to adjust the naïve estimate. It should be pointed out that the RTM could technically be removed if the characteristics of the control group are exactly the same as those of the treatment group. However, getting control group data with the exact same characteristics may not be possible in practice, as discussed in Kuo and Lord (2012). Collecting control group data usually adds extra cost and time compared to the naïve method since more data needs to be collected. (Eq. 5) ˆ CG ˆ ˆ ˆ ˆ n t i 1 j 1 N T ij 2 n t i 1 j 1 N T ij 1 n t i 1 j 1 N N C ij 2 C ij 1 Where ˆ CG = the estimate of safety effectiveness by using the control group method, ˆ = the estimated number of crashes for the treatment group in the after period, ˆ = the predicted number of crashes for the treatment group in the after period, ˆ = the estimated number of crashes for the control group in the after period, ˆ = the estimated number of crashes for the control group in the before period, N N = the observed responses for site i (T = treatment group and C = control group) and ,T 1 ij C 1 ij N N = the observed responses for site i (T = treatment group and C = control group) and ,T 2 ij C 2 ij year j (in the before period), and year j (in the after period). 19 Item-3/33 Empirical Bayes Method The EB method is recommended in the Highway Safety Manual (HSM), published by the American Association of State Highway and Transportation Officials (AASHTO) and approved for use by the FHWA (AASHTO, 2010). The HSM is a recent document that defines standardized procedures for conducting safety analyses of highway safety improvements. The EB method combines short-term observed crash numbers with crash prediction model data in order to get a more accurate estimation of long-term crash mean. The EB method is used to refine the predicted value by combining information from the site under investigation and the information from sites that have the same characteristics, such as range of traffic flow, number of lanes, lane width, etc. As an illustration, Hauer et al. (2002) use a fictional “Mr. Smith” to illustrate use of the EB method: Mr. Smith is a new driver in a city. He has no crash records during his first year of driving. Based on past crash histories for the city, a new driver in that city has 0.08 accidents per year. Based only on Mr. Smith’s record, it is not reasonable to say that he will have zero accidents or have 0.08 accidents for the next year (based on the average of all new drivers but disregarding Smith’s accident record). A reasonable estimate should be a mixture of these two values. Therefore, when estimating the safety of a specific road segment, the accident counts for this segment and the typical accident frequency of such roads are used together. The index of safety effectiveness is illustrated in Equation 6. With the EB method, the analyst first estimates a regression model or safety performance function (SPF) using the data collected with the control group. Then, the model is applied to the sites where the treatment was implemented to get a preliminary predicted value for the after period. The EB method is then used to refine the estimate to account for the RTM bias and the external factors. It is possible for the EB method to be biased if the characteristics of the treatment and control groups are not the same (Lord and Kuo, 2012). (Eq. 6) ˆ EB ˆ ˆ t n 1 i n 1 i t N 1 j T 2 ij M 1 j T ij 1 Where ˆ EB = the estimate of safety effectiveness based on the EB method; ˆ = the predicted number of crashes for the treatment group in the after period; ˆ = the estimated number of crashes for the treatment group in the after period; 1ijM = the expected responses for site i for the EB method, M W ( ) 1 (1 W) ij1 ˆ t ; ( )N ij1 W = the weight for sites for the EB method, j 1 W 1 ˆ ˆ 1 1 ; 1 ˆ = the estimate for the average number of crashes of all sites in the before period; and ˆ = the estimate of the dispersion parameter. 20 Item-3/34 1 ˆ and ˆ can be estimated using two different approaches (Hauer, 1997). They can be estimated based on a regression model or the method of moment. Both are calculated using data collected as part of the control group. For this research, the average number of crashes and dispersion parameter were estimated using a regression model. CALCULATION PROCEDURES AND EXAMPLES The EB before-after method was applied to this study with the regression models or SPFs selected from the HSM (AASHTO, 2010), which includes road types from two to five lanes. As for sites located on wider roads (six lanes and eight lanes, which are not covered in the HSM), the researchers used the SPFs from a Texas A&M Transportation Institute (TTI) study (Bonneson and Pratt, 2009). The number of crashes in each year during the before period ( was estimated using the regression model shown in Equation 7: i ) i exp( a bLn AADT ( i ) Ln L ( i )) (Eq. 7) Where i = the estimator for the average number of crashes per year for site i, ,a b = the coefficients in the regression model, AADT = the average daily traffic volume for site i, iL = the road length for site i, and Ln = natural logarithm. i Table 3 shows the regression coefficients (a, b) used in Equation 7 for multi- and single-vehicle crashes. One of the sign sites in Ohio provides an example of the detailed calculation of ,i EBM . This site is on an urban 4-lane divided highway segment in Allen County. As shown in Table 3, its intercept is -12.34 for multi-vehicle crashes and -5.05 for single-vehicle crashes, while the coefficients for the AADT are 1.36 and 0.47, respectively. For the analysis used in this report, a multi-vehicle crash is one involving two or more vehicles in the same collision. Using the EB method, the analysis procedure to get the expected number of crashes in the before period has the following steps: 1. Identify the route number and milepost by the site’s address. More specifically, the address of the example site is “1234 ABC St, Name of City, Allen County, OH.” Follow the data analysis procedures discussed in Chapter 3 to identify that the route number is 657676309 and the milepost is 7.58. 21 Item-3/35 Table 3. Coefficients for multi and single-vehicle crash regression model Regression Coefficients Crash Type Road Type* Dispersion Parameter (α) Intercept (a) AADT (b) Multi- vehicle Single- vehicle 2U 3T 4U 4D 5T 2U 3T 4U 4D −15.22 −12.4 −11.63 −12.34 −9.7 −5.47 −5.74 −7.99 −5.05 1.68 1.41 1.33 1.36 1.17 0.56 0.54 0.81 0.47 0.84 0.66 1.01 1.32 0.81 0.81 1.37 0.91 0.86 0.52 −4.82 Note: *U = undivided road, T = road with two-way left turn lane, D = divided road. 0.54 5T 2. Based on the route number and milepost obtained above, use R statistical software to select the related crashes and road files from the HSIS dataset, which includes (1) the observed crashes near the target sign site, (2) the observed crashes in the control group sites (10 sites, which are adjusted to the target sign site on the same road), and (3) the target road file, such as traffic volume, the number of lanes, and median type. For example, the number of observed crashes at the example site is 1 in 2004, and the crash counts of the related 10 control group sites are 0, 0, 1, 1, 0, 0, 0, 0, 1, and 1. The AADT of the site is 19,753 (vehicles/day), and it has four lanes. 3. Use Equation 9 to predict the crash number of the example site: )) a b Ln AADT ( Ln L ( ) exp( ( ˆ ˆ ˆ 2004 2004,multi 2004,single ˆ exp( 12.34 1.36 Ln (19753) Ln (0.2)) 0.61 exp( 5.05 0.47 Ln (19753) Ln (0.2)) 0.13 2004 2004,multi 2004,single ˆ ˆ The estimated crash counts of the site and its control group sites are 0.74 and 6.64, respectively (the estimated multi-and single crash counts of its control group are 5.36 and 1.28). 0.74 (crashes/year) 4. Due to using the SPFs from the HSM instead of the local SPFs from any existing studies conducted in the same study area, it is necessary to multiply the results by a calibration factor to adjust the prediction value (refer to Appendix A in the HSM for more details). The calibration factor of single-vehicle crashes at the example site in 2004 is 3.13, which is equal to the ratio of observed crashes in the control group divided by the predicted crash number in the control group (3.13 = (1×4+0×6)/1.28). By multiplying the above calibration factor, the final crash number estimation for the example site in 2004 should be 0.42 (=0.13×3.13). A calibration factor was calculated for each site and each year included in the study. 22 Item-3/36 5. Repeat steps 3 and 4 to get the final prediction crash number for the example site for each year in the before period. By doing so, the estimated multi- and single-vehicle crash counts of the site in 2005 are 4.65 and 0.21, respectively. Using the summary of this prediction crash number and dispersion parameter (obtained from Table 3) results in the weights (W) for this site for the multi- and single-vehicle crashes, which are 0.07 and 0.65, respectively: 1 ˆ ˆ 1 W 1 1 multiW 1 (5.43 4.65) 1 1 10.08 1 6 0.63 0.8 6. Because traffic volume and other explanatory variables may change between the before 1 0.21) 0.86 1 (0.42 0.07, 1.32 0.65 1.32 W single 1 and after periods, the researchers used one factor to account for this difference. The crash counts of the example site in 2007 and 2008 can be estimated by repeating steps 3 and 4. The estimated multi- and single-vehicle crash counts of the site in the after period are 0.84 and 0.67, respectively. Factors are estimated by: r ˆ ˆ before (0.63 / 3) (0.63 / 2) 0.67 (12.76 / 3) / (10.08 / 2) 0.84 r i multi , r i,single Also, if the time periods (Y) of the before and after periods are different, one factor is needed to adjusted it. Here, the before and after period are both two years: t 3 / 2 1.5 Y Y i i after , i before , 7. Using the EB method, the expected total number of crashes that would occur during the after ˆ i,EB M W ( ) 1 after period had the on-premise digital sign not been installed was 2.63: 0.07 10.08 (1 0.07) 0 0.65 0.63 (1 0.65) 3 0.67 1.5 1.49 0.84 1.5 1.14 (1 W) N r i i,multi,EB M M t i j 1 ) ( ij1 t i,single,EB i,all,EB M 1.14 1.49 2.63 8. The variance of the EB estimate at the example site is calculated by: Var(M ) 1,EB Var(M Var(M Var(M 1,multi,EB 1,single,EB ) 1,all,EB (1 W) M 1, t i (1 0.07) 1.14 0.84 1.5 1.31 r i EB ) ) (1 0.65) 1.49 0.67 1.5 0.54 5 1.31 0.54 1.8 9. The safety index of the example site is: t n ˆ EB ˆ ˆ 1 i n 1 i t N 1 j T 2 ij M 1 j T ij 1 9 2.63 3.43 23 Item-3/37 10. The 95 percent confidence interval of the example site is given as. ˆ Z 0.25 Var M 1, EB 3.43 1.96 1.85 0.76, 6.10 The same method was applied to other locations using the appropriate SPFs. The next chapter provides the final results of the completed safety analysis. 24 Item-3/38 CHAPTER 5: RESULTS The previous chapter explained why the research team chose to use the EB analysis procedure and provided an example of how the EB analysis was conducted. The first section of this chapter provides the results of the before-after study for each state and all the states combined. The second section provides more details about how digital on-premise signs impact traffic safety for multi-vehicle and single-vehicle crashes. The third section provides a description of an analysis of variance of the means of the safety index (θ) among the different sign characteristics such as sign color, sign size, and type of business. INDIVIDUAL AND COMBINED RESULTS As described in Chapter 3, the research team acquired the sign dataset from sign manufacturers. However, many signs were excluded from the analysis because of missing information in the dataset provided by the sign manufacturers or limitations in the HSIS crash dataset. The researchers retained only sign sites satisfying the following conditions: 1. the sign was located in Washington, North Carolina, Ohio, or California; 2. the sign was installed in 2006 or 2007 in order to have adequate time in both the before and after analysis periods to compare crash histories; and 3. the sign was located on a major road because the HSIS crash dataset usually does not include crashes that are located on minor roads or private driveways. Table 4 shows the progression in sample sizes based on sites meeting the conditions identified above. For example, the original dataset for Washington included 413 site addresses that might have an on‐premise digital sign. In order to make sure there was an adequate before‐after crash data period for further analysis, the researchers had to filter these site addresses. The first filter excluded sites where the sign was not installed in 2006 or 2007, which was needed so that there was adequate time before and after the sign was installed to perform the safety analysis. About 40 percent of the Washington sites (159 sites) met this criterion. Then, the research team used the Street View function in Google Maps to double-check whether a digital sign was present at the given addresses and whether the sign was on a major road since the HSIS crash dataset only included crashes on major roads. Only 33 sites fit this criterion. The result was that in Washington, the research team was able to use about 33 of the 400 original sites, giving an 8.0 percent yield on the raw data. Chapter 3 mentions that the main advantage of this study is the large sample size of data and advanced statistical methods that provide more accurate results than in similar studies. Figure 4 shows the sample size of this study in relation to other published papers and reports. This study has 135 sites from four states, a number much higher than the sample size of other similar studies. Hence, the results of this study are more robust and accurate. 25 Item-3/39 Table 4. Sign site sample size yield Number of Sites California Ohio Washington Included in original list from sign manufacturers Sign installation time between 2006–2007 Digital signs & located on major roads With HSIS crash data (all crashes) With HSIS crash data (multiple-vehicle crashes) With HSIS crash data (single-vehicle crashes) 86 27 6 6 6 6 North Carolina 249 94 40 33 31 32 372 178 73 63 61 63 All States 1,120 458 153 135 131 134 413 159 34 33 33 33 Data yield rate 7.0% 13.3% 16.9% 8.0% 12.1% Figure 4. A comparison of sample sizes from similar studies Table 5 presents the before-after results from the EB and the naïve statistical analysis methods. The naïve method results are provided only for comparison purposes as the naïve analysis method does not provide as meaningful results as the EB method. The results are also presented graphically in Figure 5. A safety effectiveness index (θ) of 1.0 indicates that there was no change in crashes between the before and after conditions. An index greater than 1.00 indicates that there was an increase in crash frequency in the after condition, while a value less than 1.00 indicates a decrease in crash frequency. The upper and lower bounds indicate the limits of statistical significance. If the value for is between the upper and lower bounds, then the change in crashes is not statistically significant at a 95 percent confidence level. A larger sample size usually leads to a smaller difference between the upper and lower bounds, but this may not always be the case since it is also governed by the variability observed in the data. 26 Item-3/40 Table 5. Results of statistical analysis of before-after crash condition EB Method Naïve Method Lower Bound Upper Bound Lower Bound Upper Bound State California North Carolina Ohio Washington 0.00 0.87 0.89 0.88 1.25 1.14 0.97 1.01 2.53 1.41 1.05 1.15 All states* Notes: 0.93 *“All states” represents the combined data of the four states. Naïve method values provided for comparison purposes only. 1.00 0.93 1.07 0.28 0.88 0.95 0.79 0.85 1.13 1.05 0.90 1.00 1.41 1.39 1.15 1.01 1.07 Figure 5. The safety effectiveness index and the 95 percent confidence interval for each state (all crash types) The overall results show that there is no statistically significant increase in crash frequency after installing the on‐premise digital sign because the safety effectiveness index (θ) for the entire dataset (all states) is 1.00, and the 95 percent confidence interval is 0.93–1.07 (which includes the index value of 1.00). The results for individual states are similar: no statistically significant safety impacts were observed after the installation of digital signs. In addition, one can see the width of the 95 percent confidence interval is largest for the California data. This is due to the variability of the California data and the small size of the sample set (only 6 sites). Comparing the width of the confidence intervals, from the widest to narrowest, the order is California > North Carolina > Washington > Ohio > All States. 27 Item-3/41 RESULTS FOR CRASHES RELATED TO MULTIPLE AND SINGLE VEHICLES The next analysis effort evaluated the possible safety impacts of on-premise digital signs on different types of crashes. There are several common methods to group crashes into different categories, such as the number of related vehicles, the injury levels, the collision types, and so on. Such groupings may provide some insight into the safety impacts of specific crash types, but the estimated impacts might not be precise because of a smaller sample size. The additional analysis separated crashes into two subgroups: single- and multi-vehicle crashes. All calculations and notations were the same as used previously. By using the EB method to analyze crash data related to multiple vehicles, the researchers determined that the safety effectiveness index is equal to 1.00 for all states, and the 95 percent confidence interval varies between 0.96 and 1.21. Because the confidence interval of the safety effectiveness includes 1.00, there is no statistically significant change in crash frequency after installing the on-premise digital sign. Figure 6 graphically illustrates the results for multi-vehicle crashes. The 95 percent confidence intervals are slightly larger in this figure than in Figure 5. Figure 6. The safety effectiveness index and the 95 percent confidence interval for each state (multi-vehicle crashes) The results for single-vehicle crashes are presented in Figure 7. The overall results are the similar: there are no statistically significant safety impacts from digital signs, except for California. The California results for single-vehicle crashes indicate a statistically significant decrease in crash frequency in the after period. Although the before-after results of California show a decrease in the after period, it does not affect the overall result because the low sample 28 Item-3/42 size (6 sites) makes it more difficult to establish statistical significance in the analysis results. It is also worth noting that the North Carolina data has the largest confidence interval, due to the variability in the North Carolina single-vehicle crash data. Figure 7. The safety effectiveness index and the 95 percent confidence interval for each state (single-vehicle crashes) RESULTS FOR CRASHES RELATED TO DIFFERENT TYPES OF SIGNS The research team also conducted an analysis to investigate the impacts of specific on-premise digital sign characteristics on the safety impacts of those signs. Specific sign characteristics that the research team evaluated included color (single or multi-color), size (small, medium, or large), and type of business. The research team used the analysis of variance (ANOVA) analysis method to evaluate whether the means of the safety index (θ) among the different characteristics of signs are equal. An ANOVA is one of the most common statistical methods used to compare two or more means in the analysis of experimental data. In short, ANOVA provides a statistical test of whether or not the means of multiple groups are all equal, while a t-test is suitable only for the two-group case because doing multiple two-sample t-tests would increase the risk of a Type I error (for datasets containing more than 30 observations). In addition, when there are only two means to compare, the t-test and the ANOVA are equivalent. As a result, the research team chose the one- way ANOVA as the study tool to simplify the methodology, although some digital sign characteristics, such as sign color, have only two subgroups (i.e., single color and multi-color). 29 Item-3/43 The theory of an ANOVA test is to separate the total variation in the data into a portion due to random error (sum of squares for error [SSE]) and portions due to the treatment (total sum of squares [SST]). Table 6 shows the typical form of a one-way ANOVA table. If the calculated F value (= treatment mean square [MST] / error mean square [MSE]) is significantly larger than F (k-1, N-k), the null hypothesis is rejected. F (k-1, N-k) is the critical value when the means of each group are equal. Most statistic software will also provide the corresponding p-value for researchers making their decisions in different confidence intervals. Table 6. The typical form of a one-way ANOVA table Source Treatments Error SS SST DF k-1 MS SST / (k-1) MST/MSE F P(>F) SSE N-k SSE / (N-k) SS N-1 Total (corrected) Notes: SS = sum of squares, DF = degrees of freedom, MS = mean of sum of squares, F = F-distribution (because the test statistic is the ratio of two scaled sums of squares, each of which follows a scaled chi- squared distribution), P(>F) = the p-value when the F value (= MST/MSE) is larger than F (k-1, N-k), k = number of treatments, and N = total number of cases. There are three data assumptions for applying the ANOVA method: 1. Independence: The study data are independently, identically, and normally distributed. 2. Normality: The distributions of the data or the residuals are normal. This assumption is true when the sample size is larger than 30. 3. Homogeneity of variability: Equality of variances — the variance of data between groups — should be the same. If the above conditions do not exist, the ANOVA results may not be reliable. However, if the sample size of each group is similar, one can usually ignore independence and homogeneity problems. Or statisticians may transform data (such as into the logarithmic form) to satisfy these assumptions of the ANOVA. Based on the existing sign dataset, the research team focused on three digital sign characteristics: color (single color or multi-color), sign dimension (small, medium, or large), and business type (restaurants, pharmacies and retail stores, hotels, gas stations, auto shops, or others). The definitions of sign dimension level are based on the balance principle (making the sample size of each group equal). Figure 8 shows the distribution of signs as a function of different dimensions, and the research team defined signs with an area less than 10 ft2 as small signs. The medium sign size had an area of at least 10 ft2 but no more than 15 ft2, and the large sign size had an area greater than 15 ft2. The sign size represents the area of the electronic display, not the overall size of the complete sign. It was estimated from the Street View image in Google Maps and may not be an accurate assessment of the sign dimensions. 30 Item-3/44 Figure 8. The histogram of digital signs for each sign dimension Using the ANOVA method to analyze crash data related to specific design characteristics of the sign led to the conclusion that there is no statistically significant difference among the population means of the safety effectiveness index. The following descriptions provide more detail for each of the digital sign characteristics: Color: According to images obtained from the Street View feature of Google Maps, 89 signs are single-color signs, and 37 signs are multi-colored signs. Table 7 shows the ANOVA results. The test statistic (F value) is 2.07, and its p-value is 0.1527. Because the probability is larger than the critical value (0.05 for 95 percent confidence interval), the null hypothesis of equal population means cannot be rejected. In other words, the ANOVA table shows no significant difference between the mean of safety index (θEB = crash mean in the before period/crash mean in the after period) among signs having a single color or multiple colors. ` Table 7. Analysis of variance table (color) Df Sum Sq Mean Sq F value Pr(>F) Group 1 4.464 4.4640 2.0704 0.1527 Residuals 124 267.352 2.1561 Sign dimensions: In the final sign dataset, 36 signs have a sign area less than 10 ft2, 56 signs have a sign area 10–15 ft2, and 34 signs have a sign area greater than 15 ft2. In Table 8, the F value is 0.7767, and its p-value is 0.4622. Because the probability is larger 31 Item-3/45 than the critical value (0.05 for 95 percent confidence interval), the null hypothesis of equal population means cannot be rejected. Accordingly, researchers conclude that there is no (statistically) significant difference among the population means. Table 8. Analysis of variance table (sign dimension) Sum Sq Mean Sq F value Pr(>F) Df Group 2 3.39 1.6950 0.7767 0.4622 Residuals 123 268.43 2.1823 Business type: In the final sign dataset, 7 signs are for restaurants, 18 for pharmacies and retail stores, 3 for hotels, 3 for gas stations, 7 for auto shops, and 84 for other business types. Based on Table 9, the F value is 0.5401, and its p-value is 0.7455. As with the above types, the null hypothesis of equal population means cannot be rejected because the p-value is much larger than the critical value (0.05). The sample size of some business type groups is less than 30, so the research team combined all categories of business types with less than 20 samples into one large group, the “other” category. The resulting ANOVA analysis (Table 10) provides similar results: there is no significant difference among the population means. Table 9. Analysis of variance table (six business types) Pr(>F) Sum Sq Mean Sq F value Df Group 5 5.983 1.1966 0.5401 0.7455 Residuals 120 265.833 2.2153 Table 10. Analysis of variance table (two business types) Sum Sq Mean Sq F value Pr(>F) Df Group 1 0.728 0.7289 0.333 0.5649 Residuals 123 271.088 2.18619 IMPACT OF SIGN HOLD TIME As an additional effort for this research effort, the research team worked with members of the SFI advisory panel to identify the potential impact of hold time on the relationship between on- premise digital signs and traffic safety. One of the advantages of digital signs is the ability to change the displayed message. The minimum length of time that a message must be displayed is often an element of local sign codes because some believe that frequent changing of sign messages can increase driver distraction and lead to increased crashes. Because the researchers were working with a large number of individual sites and crash records for the after period that spanned two years, it was not possible within the available resources of this project to determine what message(s) were displayed at the time of a crash or the hold time used at a particular site at the time of a crash. As a surrogate for including hold times as part of the individual site characteristics, the research team acquired information for the hold time regulations in the jurisdictions where the signs were 32 Item-3/46 located. The 135 sign sites were located in 108 jurisdictions. A member of the SFI advisory panel contacted these jurisdictions and was able to identify hold time regulations for 66 of them. The hold time regulations of these 66 jurisdictions are summarized in Table 11. Input from the advisory panel indicated that when a jurisdiction has no statutory language regarding digital sign hold times, it most often means that sign users are able to program their sign to change messages as often as they see fit. In some cases, it could mean that the state standard for digital signs applies, which ranges from 6 to 8 seconds in the four states included in the analysis. Table 11. Summary of sign hold times Minimum Hold Time Number of Jurisdictions 2–6 seconds 7–10 seconds 20 seconds 1–60 minutes 24 hours Variance required* No specific restriction Total 14 12 3 2 2 4 29 66 * Hold times were established by variance on a case-by-case basis. 33 Item-3/47 CHAPTER 6: SUMMARY AND CONCLUSIONS While there have been significant amounts of research devoted to the safety impacts of geometric design features and other aspects of the publicly owned transportation infrastructure, the same cannot be said about research on the safety impacts of privately owned signs that are directed to users of public roads. This research effort focused on addressing the safety impacts of on- premise digital signs. Previous research by others has documented the safety effects of on- and off-premise digital signs and their potential influence on crash risk to some extent. However, the results of recent crash studies are not consistent, and most studies have some important weaknesses, such as neglecting biases related to the regression-to-the-mean effects, low statistical power, and analysis results based on erroneous assumptions. In addition, Molino et al. (2009) report that the results from these studies are not comparable because of their different study methods, statistical powers, and cares of execution, which affected the quality of the research. The research effort described in this report examined the safety impacts of on-premise digital signs using a large sample size of data and advanced statistical methods that provide more accurate results than previous studies. With the help of sign data provided by sign-manufacturing companies and crash data obtained from the Federal Highway Administration Highway Safety Information System, the research team obtained extensive datasets for signs and crashes in four states. The research team began the safety analysis with 1,120 potential study sites, but only 135 sites were usable due to limitations related to the individual signs or the related crash data. Although the yield of usable data was only 11.3 percent, the final sample size of 135 sites was much higher than the sample size of other published papers and reports related to on- and off- premise signs, indicating the results of this research are more robust and accurate. The research team used the empirical Bayes (EB) statistical analysis method, which is the method recommended in the Highway Safety Manual, to conduct the safety analysis described in this report. The Highway Safety Manual is a recently published document that is recognized within the transportation profession as the authoritative document for analyzing the safety impacts of various transportation improvements or treatments. The EB analysis procedure uses a before-after approach, with the before and after values modified to address local safety characteristics, regression to the mean, and other factors. The EB method reports the safety impacts through the use of a safety index indicator (represented by ). A value greater than 1 indicates an increase in crashes, and a value less than 1 indicates a decrease in crashes from the before to the after period. However, for the results to be statistically significant, the value must be outside the limits of the 95 percentile confidence interval. For the entire sample size of 135 sites, the results from the EB method show that there is no statistically significant change in crash frequency associated with installing on‐premise digital signs because the safety effectiveness index (θ) is determined to be 1.00, and the 95 percent confidence interval is equal to 0.93 to 1.07 (which includes 1.00, indicating no statistically significant change). The research team also conducted the analysis for each of the four individual states and obtained the same results: there are no statistically significant safety impacts from 34 Item-3/48 installing on-premise digital signs. In addition, the researchers analyzed the safety impacts related to both single- and multi-vehicle crashes. The results for these analyses were also the same: there is no statistically significant increase in crashes associated with the installation of on- premise digital signs. Chapter 5 includes plots that illustrate the safety index values and confidence intervals for all of these results. As a final analysis, the research team performed an ANOVA to evaluate whether the means of the safety index (θ) varied as a function of sign factors (color, size, and type of business). The color analysis evaluated whether there was a difference in the means of the safety index for single- and multi-colored signs, and the results did not find a difference. The size analysis divided the signs in the study into three categories (<10 ft2, 10–15 ft2, and >15 ft2), and the results did not find a difference. Signs were also categorized by the type of business (restaurants, pharmacies and retail stores, hotels, gas stations, auto shops, and others). Once again, there were no differences in the means. Overall, the ANOVA analysis did not identify any factor that led to an increase or decrease in traffic safety for the subcategories evaluated in the ANOVA. Based on the analysis performed for this research effort, the authors are able to conclude that there is no statistically significant evidence that the installation of on-premise signs at the locations evaluated in this research led to an increase in crashes. 35 Item-3/49 CHAPTER 7: REFERENCES AASHTO. 2010. Highway Safety Manual. Washington, D.C. Abbess, C., D. Jarett, and C.C. Wright. 1981. Accidents at Blackspots; Estimating the Effectiveness of Remedial Treatment with Special Reference to the ‘Regression-to-Mean’ Effect. Traffic Engineering and Control, pp. 535–542. Bonneson, J.A., and M.P. Pratt. 2009. Roadway Safety Design Workbook. Research Report 0-4703-P2. Texas Transportation Institute, College Station, Texas. Danielsson, S. 1986. A Comparison of Two Methods for Estimating the Effect of a Countermeasure in the Presence of Regression Effects. Accident Analysis and Prevention 18 (1), pp. 13–23. Davis, G.A. 2000. Accident Reduction Factors and Causal Inference in Traffic Safety Studies: A Review. Accident Analysis and Prevention 32 (1), pp. 95–109. Farbry, J., K. Wochinger, T. Shafer, N. Owens, and A. Nedzesky. 2001. Research Review of Potential Safety Effects of Electronic Billboards on Driver Attention and Distraction. Report No. FHWA-RD-01-071. Federal Highway Administration, Washington, D.C. Google Earth. 2008. Available at http://www.google.com/earth/index.html [Accessed 31 August 2011]. Oxford. Hauer, E. 1980a. Bias-by-Selection: Overestimation of the Effectiveness of Safety Countermeasures Caused by the Process of Selection for Treatment. Accident Analysis and Prevention 12 (2), pp. 113–117. Hauer, E. 1980b. Selection for Treatment as a Source of Bias in Before-and-After Studies. Traffic Engineering and Control 21 (8), pp. 419–421. Hauer, E. 1997. Observational Before-After Studies in Road Safety. Elsevier Science Ltd., Hauer, E. 2005. The Road Ahead. Journal of Transportation Engineering 131(5), Institute of Transportation Engineers, Washington, D.C., pp. 333–339. Hauer, E., and B. Persaud. 1983. Common Bias in Before-and-After Accident Comparisons and Its Elimination. Transportation Research Record 905, pp. 164–174. Hauer, E., D.W. Harwood, F.M. Council, and M.S. Griffith. 2002. The Empirical Bayes Method for Estimating Safety: A Tutorial. Transportation Research Record 1784, pp. 126–131. Hauer, E., P. Byer, and H. Joksch. 1983. Bias-by-Selection: The Accuracy of an Unbiased Estimator. Accident Analysis and Prevention 15 (5), pp. 323–328. Hughes, P.K., and B.L. Cole. 1986. What Attracts Attention When Driving? Ergonomics 29 (3), pp. 377–391. Kuo, P.-F., and D. Lord. 2012. Accounting for Site-Selection Bias in Before-After Studies for Continuous Distributions: Characteristics and Application Using Speed Data. Paper 12-2070, 91st Annual Meeting of the Transportation Research Board. Lee, S.E., M.J. McElheny, and R. Gibbons. 2007. Driving Performance and Digital Billboards. Center for Automotive Safety Research, Virginia Tech Transportation Institute, Blackburg, Virginia. Li, W., A. Carriquiry, M. Pawlovich, and T. Welch. 2008. The Choice of Statistical Models in Road Safety Countermeasure Effectiveness Studies in Iowa. Accident Analysis and Prevention 40 (4), pp. 1531–1542. 36 Item-3/50 Lord, D., and P.-F. Kuo. 2012. Examining the Effects of Site Selection Criteria for Evaluating the Effectiveness of Traffic Safety Improvement Countermeasures. Accident Analysis and Prevention 42, pp. 52–63. Lord, D., and F. Mannering. 2010. The Statistical Analysis of Crash-Frequency Data: A Review and Assessment of Methodological Alternatives. Transportation Research - Part A 44 (5), pp. 291-305. Mace, D. 2001. Chapter 2: On-Premise Signs and Traffic Safety. In Context-Sensitive Signage Design, American Planning Association, Chicago, Illinois. Maher, M., and L. Mountain. 2009. The Sensitivity of Estimates of Regression to the Mean. Accident Analysis and Prevention 41 (4), pp. 861–868. Miranda-Moreno, L.F. 2006. Statistical Models and Methods for Identifying Hazardous Locations for Safety Improvements. Ph.D. thesis, University of Waterloo. Molino, J.A., J. Wachtel, J.E. Farbry, M.B. Hermosillo, and T.M. Granda. 2009. The Effects of Commercial Electronic Variable Message Signs (CEVMS) on Driver Attention and Distraction: An Update. Report No. FHWA-HRT-09-018. Federal Highway Administration, Washington, D.C. Noland, R.B. 2003. Traffic Fatalities and Injuries: The Effect of Changes in Infrastructure and Other Trends. Accident Analysis and Prevention 35 (4), pp. 599–611. Persaud, B., and C. Lyon. 2007. Empirical Bayes Before-After Safety Studies: Lessons Learned from Two Decades of Experience and Future Directions. Accident Analysis and Prevention 39 (3), pp. 546–555. Persaud, B.N., R. Retting, P. Garder, and D. Lord. 2001. Observational Before-After Study of U.S. Roundabout Conversions Using the Empirical Bayes Method. Transportation Research Record 1751, pp. 1–8. Quddus, M.A. 2008. Modeling Area-Wide Count Outcomes with Spatial Correlation and Heterogeneity: An Analysis of London Crash Data. Accident Analysis and Prevention 40 (4), pp. 1486–1497. Smiley, A., B. Persaud, G. Bahar, C. Mollett, C. Lyon, T. Smahel, and W.L. Kelman. 2005. Traffic Safety Evaluation of Video Advertising Signs. Transportation Research Record: Journal of the Transportation Research Board 1937, pp. 105–112. Tantala, A.M., Sr., and M.W. Tantala. 2007. A Study of the Relationship between Digital Billboards and Traffic Safety in Cuyahoga County, Ohio. Foundation for Outdoor Advertising Research and Education, Washington, D.C. Tantala, A.M., Sr., and M.W. Tantala. 2009. An Update of a Study of the Relationship between Digital Billboards and Traffic Safety in Cuyahoga County, Ohio. Foundation for Outdoor Advertising Research and Education, Washington, D.C. Tarko, A., S. Eranky, and K. Sinha. 1998. Methodological Considerations in the Development and Use of Crash Reduction Factors. Preprint Paper (unpublished) at 77th Annual Meeting of the Transportation Research Board, Washington, D.C. U.S. Small Business Administration. 2003. The Signage Sourcebook: A Signage Handbook. U.S. Small Business Administration, Washington, D.C., and the Signage Foundation for Communication Excellence, Inc., Sherwood, Oregon. Wachtel, J. 2009. Safety Impacts of the Emerging Digital Display Technology for Outdoor Advertising Signs. Final Report under NCHRP Project 20-7 (256). Transportation Research Board, Washington, D.C. 37 Item-3/51 Wachtel, J., and R. Netherton. 1980. Safety and Environmental Design Considerations in the Use of Commercial Electronic Variable-Message Signage. Report No. FHWARD-80-051. Federal Highway Administration, Washington, D.C. Wisconsin Department of Transportation. 1994. Milwaukee County Stadium Variable Message Sign Study: Impacts of an Advertising Variable Message Sign on Freeway Traffic. Internal Report (unpublished). District 2, Freeway Operations Unit. Wright, C., C. Abbess, and D. Jarrett. 1988. Estimating the Regression-to-Mean Effect Associated with Road Accident Black Spot Treatment: Towards a More Realistic Approach. Accident Analysis and Prevention 20, pp. 199–214. Ye, Z., and D. Lord. 2009. Estimating the Variance in Before-After Studies. Journal of Safety Research 40 (4), pp. 257–263. Ye, Z., D. Veneziano, and D. Lord. 2011. Safety Impact of Gateway Monuments. Accident Analysis and Prevention 43 (1), pp. 290–300 38 Item-3/52 APPENDIX A: STEP-BY-STEP INSTRUCTIONS FOR STUDENTS TO RECORD SIGN DATA 1. Open one SFI sign dataset (e.g., “Washington_2006-2007.xls”). This dataset includes about 150 signs located in the state of Washington during 2006–2007. 2. Input the address information (such as Primary Street Address, City, ZIP Code, County Name, and State) of each sign in Google Maps and use the Street View function to identify the target signs. Please see this link, http://maps.google.com/help/maps/starthere/index.html#streetview&utm_campaign=en& utm_medium=et&utm_source=en-et-na-us-gns-svn&utm_term=gallery, for a demo about how to use the Street View. If you did not find any on-premise digital signs near this site, please make a note in Table 12. Check the characteristics of each sign (including colors, dimensions, and business types) and fill out Table 12. Then, use the “Print Screen” button to copy each sign’s picture, and paste it in this document (such as Figure 9). The different business types are classified as (1) Restaurant, (2) Pharmacy and Retail Store, (3) Hotel, (4) Gas Station, (5) Auto Shop, and (6) Other. Table 12. Example work table of site data collection procedure Google Earth Google Maps Sign ID Address Installation Date Picture Color (Single/ Multi.) Dimension (Estimated) Business Type County ID Route # Distance Note Mile- post 2006/9/15 Fig 2 S 3 ft × 6 ft 6 Mason (23) 101 19.3 335.72 79016 19330 N US HIGHWAY 101 Shelton 98584 Mason County, WA 3. Then, use Google Earth to determine the county and route number, and to measure the distance between the closet county boundaries and sign location along the route (recorded in the distance column). The corresponding ID for county and route number is based on the HSIS data manual (file name: guidebook_WA[1].pdf). Then, estimate the milepost value of the sign by the distance and the milepost of the route in the boundaries (based on the HSIS road file, such as wa04road.xls). Take Figure 10; for example, the end mile point of Highway 101 in the county boundary is 355.18, and the distance between the sign and the county boundary is 19.3; so, the milepost of our sign is 335.72. Generally, the milepost value increases from south to north and from west to east. However, the best way to check it is to compare the value of the milepost of adjusted counties. For example, the milepost of US 101 in Mason County is 313.96~355.18, and the milepost of US 101 in Thurston County (located south of Mason) is 355.18~365.56. So, it is known that the mileposts increase from north to south in Mason County. The above variables will be used in the R software to select target crashes from HSIS crash datasets. 4. Write down any questions or comments in the note column. Feel free to ask us if you have any questions. 39 Item-3/53 Figure 9. Example screenshot of Google Maps Figure 10. Example screenshot of Google Earth 40 Item-3/54 APPENDIX B: STATISTICAL SYMBOLS The following statistical symbols are used throughout this report. = the safety effectiveness, 0 (cid:3407) (cid:2016) (cid:3409) 1 (can be theoretically higher, but not in this study). n = the sample size. = the dispersion parameter (of the negative binomial model). t = the time period. ˆ CS = the estimate of safety effectiveness by using the CS method. ˆ naive = the estimate of safety effectiveness by using the naïve method. ˆ CG = the estimate of safety effectiveness by using the control group method. ˆ EB = the estimate of safety effectiveness by using the EB method. ˆ = the estimated number of crashes for the treatment group in the after period. ˆ = the estimated number of crashes for the treatment group in the before period. ˆ = the estimated number of crashes for the control group in the after period. ˆ = the estimated number of crashes for the control group in the before period. N N = the observed responses for site i (T = treatment group and C = control group) and N N = the observed responses for site i (T = treatment group and C = control group) and ,T ij 1 ,T ij 2 C ij 1 year j (in the before period). C ij 2 year j (in the after period). 1ijM = the expected responses for site i for the EB method, M W ( ) 1 (1 W) ij1 ˆ ( )N ij1 t j 1 . W = the weight for sites for the EB method, W 1 ˆ ˆ 1 1 . 1 ˆ = the estimate for the average crash rate of all sites in the before period. ˆ = the estimate of the dispersion parameter (from the negative binomial model). 41 Item-3/55 SIGN RESEARCH FOUNDATION EXECUTIVE SUMMARY DIGITAL SIGNAGE AND TRAFFIC SAFETY: A STATISTICAL ANALYSIS Item-3/56 W hen it comes to studying the impact of digital signage on traffic safety, a simple measure of crash frequencies before and after sign installation doesn’t provide a complete picture. That’s why a group of researchers from Texas A&M University incorporated the Empirical Bayes (EB) statistical analysis method, allowing for better control of external factors. Their work is believed to be the first comprehensive and scientifically based research on the issue. The safety analysis included 135 sign locations based on the following criteria: ■ ■ ■ The signs were located in Washington, North Carolina, Ohio or California. The signs were installed in 2006 or 2007 in order to have adequate time in both the before and after analysis periods to compare crash histories. The signs were located on major roads, as the Highway Safety Information System (HSIS) crash dataset usually does not include crashes that are located on minor roads or private driveways. As the use of digital on-premise signs continues to increase, so, too, have efforts to regulate the way digital messages are displayed. Jurisdictions often cite traffic safety impact as justification for local sign codes and ordinances. This research, however, provides a scientifically based, national analysis to help all involved better understand the true impact of on-premise digital signage on safety. The 2012 study was performed by H. Gene Hawkins, Jr., Ph.D., P.E., associate professor and research engineer, Zachry Department of Engineering, Texas A&M University; Pei- Fen Kuo, graduate research assistant, Texas A&M Transportation Institute; and Dominique Lord, Ph.D., associate professor and research engineer, Zachry Department of Engineering, Texas A&M University. It was funded by a grant from the Sign Research Foundation. 1 BEST PRACTICES This executive summary highlights three key takeaways from the report. THE STUDY USES ADVANCED STATISTICAL METHODS ALONG WITH A LARGE SAMPLE SIZE OF DATA. ■ ■ ■ ■ The 135 sites—narrowed down from an original pool of 3,000 possible locations—came from four states. Sites had to meet all study criteria of location and date of installation to be included. The size and specificity of the study allowed for more robust and accurate results than previous research. Crash data was acquired from the Federal Highway Administration’s Highway Safety Information System. The data also included roadway characteristics such as number of lanes, speed limits and other factors. The research team gathered information about the location of on-premise digital signs from two sign manufacturing companies. The datasets were then carefully merged through a lengthy validation process. A basic comparison of crash frequencies before and after sign installation would be known as a naïve before-after analysis. The Empirical Bayes method used for the study, however, is the recommended procedure for evaluating the impact of safety treatments. Safety impacts are represented by the safety index, a ratio of safety before and after an event, in this case, the installation of digital signs. THE DATA REVEALED THAT THERE IS NO STATISTICALLY SIGNIFICANT INCREASE IN CRASH FREQUENCY AFTER INSTALLING AN ON-PREMISE DIGITAL SIGN. ■ The research involved a review of previous studies, including hypotheses that on-premise business signs distract drivers, leading to higher crash rates, and that on-premise business signs might mask the visibility of regulatory and warning road signs, also increasing crash risk. Previous research has been inconsistent, and some studies have had important weaknesses and study limitations. Hawkins, Kuo and Lord took every avenue to deliver as accurate and valid a study as possible; the analysis even separated crashes into single- and multi-use incidents. No statistically significant differences were found overall. The safety index for all four states was 1.0 with a 95 percent confidence interval that ranged from 0.93 to 1.07, meaning the use of on-premise digital signs does not increase the risk of crashes. ■ ■ 2 ■ Item-3/57 FIVE DISTINGUISHING FACTORS Thorough literature review on safety aspects of digital signage and previous statistical analyses 1 2 Data collection of sign installation dates & crash data over four states (largest sample size of any study to date) 3 5 Key study findings, conclusions, and recommendations Evaluation and gap analysis of past studies Use of the Empirical Bayes (EB) statistical analysis method - recommended by the Highway Safety Manual, the “authoritative document for analyzing the safety impacts of various transportation improvements or treatments.” 4 USING THE ANALYSIS OF VARIANCE (ANOVA), THE TEAM ALSO EXPLORED THE IMPACT OF ON-PREMISE DIGITAL SIGN COLOR, DIMENSION AND BUSINESS TYPE ON SAFETY. ANOVA is a collection of statistical models used for comparing several groups or variables for statistical significance. 3 Hawkins, Kuo and Lord compared single color and multi-color signs as part of the study; 89 were single and 37, multi-colored. They found no significant difference in the mean of safety index among signs having single or multiple colors. The study considered small, medium and large signs. In the final dataset, 36 signs had a sign area of less than 10 ft2; 56 had a sign area from 10-15 ft2; and 34 had a sign area greater than 15 ft2. Sign size was delineated based on the area of the electronic display, not the overall size of the complete sign. Here, too, the researchers found no statistically significant difference among the categories in terms of safety. ■ ■ ■ ■ The researchers looked at business type, such as restaurants (seven); pharmacies and retail stores (18); hotels (three); gas stations (three); auto shops (seven); and others (84). Once again, there was no statistically significant difference among the population means. Read the Full Report Statistical Analysis of the Relationship between On-Premise Digital Signage and Traffic Safety (2012): www.signresearch.org/trafficsafety Item-3/58 DID YOU FIND THIS SUMMARY USEFUL? SUPPORT MORE RESEARCH LIKE THIS WITH A DONATION. GIVETOSRF.ORG FOLLOW US @SIGNRESEARCH PUBLISHED BY: WWW.SIGNRESEARCH.ORG This report remains the property of the Sign Research Foundation. None of the information contained within can be used, republished or reprinted without express permission from the Sign Research Foundation. Item-3/59 February 19, 2018 Subject: Brightness settings Watchfire full color led signs are designed for a maximum day brightness of 10,000 NITs and a maximum night brightness of 750 NITs. Monochrome led signs are designed for a maximum day brightness of 5,000 and 500 at night. (Adjustable to any value). Typical brightness is 30% of maximum depending on the message content. Watchfire continues to work with Sign Associations such as USSC, ISA, and OAAA regarding their findings on safe effective day and nighttime brightness levels for led signs. Watchfire is committed to the safety of the communities in which we sell our products, by providing a safe means of advertising. To be considered “safe” led signs must have acceptable legibility and avoid over brightness. A field test is recommended as led sign readability can be affected by existing ambient light levels. Watchfire signs offers zoning friendly software to comply with local codes. Upon request the software can be preset to match the brightness, hold time, and transition settings mandated by those codes. Brightness Controls • The software uses what is referred to as a software photocell and relies on zip code location of the sign to determine the longitude and latitude. The sign is appropriately dimed or brightened based around seasonally adjusted sunrise and sunset times. • An actual photocell is available to control brightness. When present the photocell takes priority over the software photocell. It responds to actual light conditions. Item-3/60 Conformance of Watchfire Signs to OAAA/ISA Lighting Guidelines The Outdoor Advertising Association of America (OAAA) and the International Sign Association (ISA) have recommended criteria for brightness standards for outdoor LED signs. Watchfire by Time-O-Matic has validated that all products can comply with these guidelines. The guidelines highlight the importance of ensuring appropriate brightness settings. Energy conservation, community goodwill, and extending the life expectancy of the displays are all goals of the criteria. All Watchfire signs incorporate a sunrise/sunset calculation algorithm that uses the sign’s latitude and longitude coordinates to correctly dim the sign at dusk and brighten it at dawn in several steps. An additional photocell option is available that will also allow the sign to dim in response to severe overcast weather during the day. LED sign brightness is usually specified in NITs or candelas per square meter. The instruments for measuring NITs are expensive and not commonly available to sign companies or communities. The new guidelines offer a means to specify the sign brightness using a foot- candle meter, which is relatively inexpensive and more available. Watchfire signs ship with the night brightness settings between 500-750NITs depending on model. The day maximum brightness is between 7500 and 11000 NITs depending on model. During normal operation an average of 30-40% of the pixels are on, so normal brightness is less than half of the maximum on average. The Brightness guidelines specify that the signs will not be brighter than 0.3 foot candles than the ambient light levels at night at a specified distance. The distance depends on the size of the sign. Since the specification is relative to conditions, the brightness of the surrounding area is taken into consideration. In most cases, the default settings on Watchfire signs meet these guidelines. Areas with little or no ambient lighting may require additional manual dimming. This is a simple setting that can be made at the factory or in the field to any sign manufactured by Watchfire past or present. The settings are password-protected. The user cannot change the setting without assistance from Time-O-Matic Support. The default day settings of Watchfire signs also fall under the guidelines which indicate that 7500-10000 NITs is needed to overcome the full sunlight. Watchfire models that ship with higher day settings can be adjusted down if necessary. The detailed ISA specification and measurement method may be downloaded from the ISA website at this link: http://www.signs.org/IndustryResources/TechnicalRegulatoryResources/BrightnessGuideforEle ctronicMessageCenters/tabid/745/Default.aspx Item-3/61 H C R A E S E R Night-time Brightness Level Recommendations for On-Premise Electronic Message Centers Updated August 2016 PRODUCED BY: Item-3/62 [Blank inside front cover] Item-3/63 ISA RESEARCH TABLE OF CONTENTS INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 CASE STUDIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 RECOMMENDED LANGUAGE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 SIX STEPS: EMC BRIGHTNESS LEVELS WITH OPERATIONAL CONTROL . . . . . . . . . . . . . . . . . . . . . . . . . 9 WITHOUT OPERATIONAL CONTROL . . . . . . . . . . . . . . . . . . . . . 11 LEARN MORE ABOUT EMCS The International Sign Association offers an Electronic Message Center (EMC) Resource Center, with resources on: • EMCs and traffic safety • A framework for developing EMC sign code language • The differences between EMCs and digital billboards www.signs.org/local ADDITIONAL SIGN CODE RESOURCES The International Sign Association has developed numerous tools to help communities develop better sign codes. All are housed at www.signs.org/local, including: • The Supreme Court ruling, Reed v. Town of Gilbert • Model sign codes • Best practices in regulating temporary and wayfinding signs • The Economic Impact of On-Premise Signs ISA’s advocacy team is available to provide complimentary assistance on sign codes and sign-related issues. Contact SignHelp@signs.org or 703.836.4012. © International Sign Association 1 Item-3/64 ISA RESEARCH INTRODUCTION ELECTRONIC MESSAGE CENTERS (EMCs) Electronic message centers, or EMCs, continue to grow in popularity for business and community use. You may have heard EMCs being referred to as changeable message displays or digital signs. EMCs are not digital billboards, which advertise a good or service that is located away from the sign. Rather, EMCs are digital signs that are located on the premises, and that advertise goods and services that are available at the location. Electronic Message Center (EMC)/on-premise sign advertising a bank that is located on the same premises as the sign Digital billboard/off-premise sign advertising an automobile business in another location There is often confusion regarding on- and off-premise digital signs. However, EMCs and digital billboards have very distinct capabilities and purposes, each targets a specific audience and each has traditionally been treated under separate legal and regulatory regimes, a zoning practice which was noted in the 2015 U.S. Supreme Court ruling in Reed v. the Town of Gilbert. For the purposes of this publication, we are focusing solely and exclusively on EMCs. EMCs that are too bright at night can be offensive and ineffective. Night-time EMC brightness is an issue where sign users, the sign industry, and local offices have a common goal: ensuring that EMCs are appropriately legible. We know the messages that these signs convey can be rendered unattractive and perhaps even unreadable if they are programmed too bright. That’s why many sign companies recommend to their customers that in order for these signs to be most effective, their brightness be set at such a level to be visible, readable and conspicuous. 2 © International Sign Association Item-3/65 ISA RESEARCH The International Sign Association (ISA) retained noted lighting expert Dr. Ian Lewin of Lighting Sciences to help the industry develop scientifically- researched, understandable recommendations for EMC brightness. Dr. Lewin was a past chair of the Illuminating Engineering Society of North America (IES), and was greatly respected within the lighting field. His work for ISA was conducted with the input of experts within the sign industry. As a result of his research, Dr. Lewin recommended two different brightness settings based on whether the EMC was located in an area of high or low ambient light. After field testing and utilizing Dr. Lewin’s recommendations, it was determined that using the more conservative recommendation is appropriate in areas of both low and high ambient light. In order to simplify Dr. Lewin’s recommendations, and to take a more reasonable approach to ensure that EMCs are sufficiently visible but not overly bright, it is recommended that EMCs not exceed 0.3 footcandles over ambient lighting conditions when measured at the recommended distance, based on the EMC size. The research and the recommendations contained in this report pertain only to EMCs, not traditionally internally illuminated signs, such as these channel letter and neon signs below. EMCs use a different lighting technology than most of these types of signs, and as such the scientific approach differs. Community leaders should understand that, while it is recommended that brightness measurements be taken perpendicular to the sign, sign viewers rarely see the sign at that same perpendicular approach. At any viewing point away from or off the forward angle, the apparent brightness will be reduced. In other words, the measurements will capture the recommended brightness levels, but, unless viewers are looking at the sign directly perpendicular, they will not perceive the brightness at the full level. We have provided recommended statutory language and tips to measure brightness with and without control of the EMC. If you need further assistance, feel free to contact ISA, signhelp@signs.org or at (703) 836-4012 to answer any of your EMC questions. FOOTCANDLES VS. NITS: WHICH MEASUREMENT IS BETTER? This document recommends communities adopt illumination measurements in footcandles as compared to nits. Here are a few reasons why more than 200 localities and many state departments of transportation have adopted the footcandle measurement for EMCs: FOOTCANDLES Measures illuminance Accounts for ambient light conditions Luxmeter measuring device $100 “Twilight” measurement possible Measures light impact and appearance Works with roadway lighting standards Easier to check and enforce * While the main advantage of using nits as compared to footcandles is that daytime measurement is possible, EMC brightness is typically more of an issue at night. NITS Measures luminance Measures only the amount of brightness emitted Luminance spectrometer (nit gun) - $1,000 Does not allow adjustment based on ambient light Does not measure appearance Difficult to measure accurately Difficult to enforce © International Sign Association 3 Item-3/66 ISA RESEARCH CASE STUDY: Columbus, Ohio COMMUNITY . . . . . . . . . . . . Columbus, Ohio POPULATION . . . . . . . . . . . . 836,000 LOCATION . . . . . . . . . . . . . . As Ohio’s largest city and state capitol, Columbus is the country’s 15th largest city. SPECIFIC EMC ISSUE . . . . . Crafting a reasonable, enforceable code that addresses complaints while preserving the ability for businesses to use what it termed automatic changeable copy signs. As automatic changing copy signs—as Columbus refers to EMCs—grew in use, so did community complaints. By 2011, city planners began to edit the graphics codes to limit special effects. The goal was to continue to allow for a variety of commercial graphics, “but not at the expense of neighborhoods,” said Lisa Russell, the city’s Planner II who facilitated the code development project. The city had in place certain limits on automatic changing copy signs, aka EMCs, in the graphics code, limiting their use to commercial and manufacturing zoning districts and requiring that only half of the sign could be used for the changeable copy. But signs lacked brightness limits and a hold time. Russell led a team to draft the new code, which incorporated a brightness limit for both on-premise and off-premise signs. The testing method also is included in the code. It was the result of much scientific discussion. “I believe that the best answer is revealed if you have enough information,” Russell said. The committee included a community group leader who was an architect specializing in lighting and representatives from the sign and graphics industry. “When we started exploring brightness, it appeared the footcandle method was the way to go,” Russell said. “However, some group members wanted us to explore the luminance method. ISA believed so strongly that the luminance method was problematic that they brought a demonstration to us.” The demonstration included a field trip to visit a sign to show the impact of the two measurement methods. “They wanted to make sure that we didn’t go down the wrong path. They rented a lift and showed us that with the luminance method you’d have to get up in the lift, raise it and shine the nit gun at the sign. With the footcandle meter, you can stand on the ground.” Russell helped the group to see that the “members of the professional sign and graphics industry are not the same as end-users of signs, such as an owner of a carryout who wants to draw attention to his shop over others. We all had an interest in developing reasonable regulations instead of just banning these signs. We also did not want to take away the rights that businesses had to display electronic signs.” The new code has significantly lessened complaints about sign brightness. And when a complaint is received, the code enforcement officers have a verifiable process for determining whether the sign complies with the code. 4 © International Sign Association Item-3/67 CASE STUDY: Kitsap County, Washington ISA RESEARCH COMMUNITY . . . . . . . . . . . Kitsap County, Washington POPULATION . . . . . . . . . . . . 260,000 LOCATION . . . . . . . . . . . . . . Across the Puget Sound from Seattle and bordered by rural SPECIFIC EMC ISSUE . . . . . Existing codes did not cover electronic signs. communities on the west. It is the third most densely populated county in the state. As a “transition” county between rural Washington and the metropolitan city of Seattle, Kitsap County had the challenges of creating regulations for electronic signs that fit the county’s dual personalities. “The first step was to identify where these signs would be allowed,” said Darren Gurnee, a planner with the county. “We wanted to make sure these were restricted to areas of increased density and primarily non-residential use such as industrial zones and commercial zones within the urban growth area.” Previously, the county had allowed electronic signs “as a matter of interpretation,” Gurnee said. Crafting more defined electronic sign regulations would provide a measure of stability—and help business owners know what was allowed and where. An added bonus: Gurnee felt the signs would be more attractive than the block letters signs that had to be changed manually. While the county wanted to make it easier for businesses to convert existing static monument signs into electronic signs, it also wanted to ensure that the regulations were not written in a way that would allow billboards to convert. “We were able to craft our regulations in a way that required signs be brought into conformance before any change could be made,” Gurnee said. “Billboards were non-conforming, so that would not be an issue.” ISA provided Gurnee with industry standards—contained in this publication—and some background on the technology that today’s electronic signs offer, such as automatic dimming. It also incorporated some of the recommended language on animation, hold times and transitions. “The regulation is written in a way that it would be easy to enforce,” Gurnee said, and easy to understand, without the ambiguities contained in the previous method. The ending code created a perfect fit for both of the community’s personalities. © International Sign Association 5 Item-3/68 ISA RESEARCH CASE STUDY: SPARKS, NEVADA COMMUNITY . . . . . . . . . . . Sparks, Nevada POPULATION . . . . . . . . . . . . 93,500 LOCATION . . . . . . . . . . . . . . A rapidly growing community, Sparks is located near Lake Tahoe, SPECIFIC EMC ISSUE . . . . . Existing regulations were difficult to enforce and outdated. California, and Reno, Nevada, and is Nevada’s fifth largest city. Sparks, Nevada had existing regulations of electronic message centers—or electronic variable signs as the community deemed them. But “it wasn’t very explicit,” said senior planner Karen Melby. “The brightness standards were in lumens, which we didn’t even know how to measure.” The regulations were outdated as well—having been drafted in 2002. Technology had changed dramatically and the costs of EMCs had dropped, putting them in the range of more businesses’ budgets. “We felt we could see more coming and felt that we needed to get a handle on it.” As a first step, planners required that those seeking an EMC permit meet their standards before approval was granted, but nothing was written into the code. That method can create problems. So Melby led the city through the code revision process. She sought out industry expertise from both the planning community and the sign and graphics industry. For industry insight, she turned to ISA. ISA provided feedback on how other communities were regulating electronic message centers, and rec- ommendations on what was working for these communities. One outside group felt strongly that the standards should be regulated in nits, not footcandles. They brought in an expert who opposed the proposed regulations. But Melby held strong on the issue of footcandles. “In my research, it seems like footcandle is what you can see with your eyes while a nit is pinpointing a spot on a sign. When you look at a sign, you’re looking at the whole thing, not just one small spot.” The city adopted the widely recognized standard of 0.3 footcandles above ambient light, using the distance measurements outlined in this publication. Melby took that table, determined the formula and wrote the formula into the code. The community allows smaller signs—those under 32 square feet—to include scrolling, while those larger do not. The result has been a city that has successfully navigated the balance between business interests and community aesthetics. “We’ve had very few complaints,” Melby said. “When we do get a complaint about a sign being too bright, we go out and measure it. When they bring it down to standards, we don’t get complaints.” Being able to use a simple light meter to measure brightness is far easier than simply guessing whether the sign is in compliance, Melby said. “The other method (measuring nits) was really based on opinion. What may seem bright to me may not seem bright to you. Now, we can say, ‘This is what the meter says.’” By having clear standards that are easier to enforce, both community and business win. 6 © International Sign Association Item-3/69 EXECUTIVE SUMMARY ISA ELECTRONIC MESSAGE CENTER NIGHT-TIME BRIGHTNESS RECOMMENDATIONS This summary has been developed with an understanding that EMCs that are unreasonably bright are not effective for the communities or end users. This intends to help communities and stakeholders develop brightness standards for on-premise EMCs. The summary comprises: ISA RESEARCH 1) An overview of the importance of ensuring appropriate brightness, 2) Technology utilized to ensure appropriate brightness, and 3) Recommended brightness standards 1. Overview of the importance of ensuring appropriate night-time brightness. EMCs that are too bright at night can be offensive and ineffective. There are significant advantages to ensuring than an electronic display is not overly bright. These advantages include: » Conservation of energy » Increased life expectancy of the electronic display components » Building goodwill with the community » Ensuring the legibility of the display It is in the best interest of all stakeholders to ensure that EMCs are sufficiently bright to ensure clear legibility, while at the same time avoiding a display that is overly bright. 2. Technology utilized to ensure appropriate brightness. Most EMCs are designed to produce sufficient brightness to ensure clear legibility during daylight hours. However, daytime brightness settings are usually inappropriate for night-time viewing. The following general methods are used to dim an EMC for appropriate night-time viewing: 1. Manual Dimming. Using this method, the sign operator dims the display in response to changing ambient light conditions. 2. Scheduled Dimming. Sunset-sunrise tables allow an EMC to be programmed to dim at the same time that the sun sets and rises. This method is generally acceptable, but is more effective when used as a backup to automatic dimming controls capability, such as photocell technology. 3. Photocell Technology. An EMC that utilizes photocell technology can automatically dim as light conditions change. A photocell sensor alerts the display to adjust brightness according to ambient light conditions. 3. Recommended night-time brightness standards. Dr. Lewin recommended the development of brightness criteria based on the Illuminating Engineering Society’s (IES) well-established standards pertaining to light trespass, IES Publication TM-11-00. The theory of light trespass is based on the concept of determining the amount of light that can spill over (or “trespass”) into an adjacent area without being offensive. In order to simplify Dr. Lewin’s recommendations, and to take a more reasonable approach to ensure that EMCs are sufficiently visible but not overly bright, it is recommended that EMCs not exceed 0.3 footcandles over ambient lighting conditions when measured at the recommended distance, based on the EMC size. Email signhelp@signs.org to receive Dr. Lewin’s original research. ...it is recommended that EMCs not exceed 0.3 footcandles over ambient lighting conditions when measured at the recommended distance, based on the EMC size. © International Sign Association 7 Item-3/70 SIGN AREA VERSUS MEASUREMENT DISTANCE AREA OF SIGN sq. ft. MEASUREMENT (ft.) ISA RESEARCH RECOMMENDED LEGISLATIVE LANGUAGE “ Electronic Message Center (EMC) Criteria: The night-time illumination of an EMC shall conform with the criteria set forth in this section. A. EMC Illumination Measurement Criteria: The illuminance of an EMC shall be measured with an illuminance meter set to measure footcandles accurate to at least two decimals. Illuminance shall be measured with the EMC off, and again with the EMC displaying a white image for a full color-capable EMC, or a solid message for a single-color EMC. All measurements shall be taken as close as practical to a perpendicular plane of the sign at the distance determined by the total square footage of the EMC as set forth in the accompanying Sign Area of a Sign versus Measurement Distance table. B. EMC Illumination Limits: The difference between the off and solid-message measurements using the EMC Measurement Criteria shall not exceed 0.3 footcandles at night. C. Dimming Capabilities: All permitted EMCs shall be equipped with a sensor or other device that automatically determines the ambient illumination and programmed to automatically dim according to ambient light conditions, or that can be adjusted to comply with the 0.3 footcandle measurements. D. Definition of EMC: A sign that utilizes computer-generated messages or some other electronic means of changing copy. These signs include displays using incandescent lamps, LEDs, LCDs or a flipper matrix. 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 110 120 130 140 150 160 170 180 190 200 220 240 260 280 300 ” 32 39 45 50 55 59 63 67 71 74 77 81 84 87 89 92 95 97 100 105 110 114 118 122 126 130 134 138 141 148 155 161 167 173 * For signs with an area in square feet other than those specifically listed in the table (i.e., 12 sq ft, 400 sq ft, etc), the measurement distance may be calculated with the following formula: Measurement Distance = Area of Sign Sq. Ft. x 100 8 © International Sign Association Item-3/71 HOW TO MEASURE THE NIGHT-TIME BRIGHTNESS OF AN EMC WITH OPERATIONAL CONTROL (Note: This method can be completed by one individual, but requires operational control to shutoff the EMC) ISA RESEARCH STEP 1 OBTAIN AN ILLUMINANCE METER. STEP 3 DETERMINE THE MEASUREMENT DISTANCE. Using the total square footage found in Step 2, look up the measurement distance in the table provided in the Recommended Legislative Language on page 8, to determine the distance to measure the brightness of the EMC. The distance should be measured perpendicular to the EMC sign face. The use of a measuring wheel, laser finder or a smartphone app are the most convenient ways to measure the distance. Purchase or otherwise procure an illuminance meter. Most city/county traffic departments have an illuminance meter, which are also referred to as lux or footcandle meters (lux is the metric measure of illuminance; footcandles is the English measure of illuminance). The illuminance meter must have the ability to provide a reading up to two decimal places and must be set to read footcandles. It is preferred to have an illuminance meter with a screw-mount that allows the sensor to be mounted on a tripod. A tripod ensures that the highly sensitive sensor is held perfectly still; otherwise it may be difficult to obtain an accurate reading. STEP 2 DETERMINE SQUARE FOOTAGE. Determine the square footage of the face of the electronic message sign (EMC) by multiplying the height and width of the EMC. This infor- mation may be available in a permit application, or can be determined by physically measuring the height and width of the EMC. Do not include the sign face square footage attributable to any additional static signs associated with the EMC (if applicable). © International Sign Association 9 Item-3/72 STEP 6 ENSURE THAT THE DISPLAY CAN ADJUST TO DIFFERENT AMBIENT CONDITIONS. Inspect the sign to ensure that it incorporates a photocell or other technology to ensure that the display can adjust according to ambient lighting conditions. ISA RESEARCH STEP 4 PREPARE THE DISPLAY FOR TESTING. Ensure that the EMC is programmed to alternate between a solid white (or in the case of a monochrome display – the solid color of the display) message and a blank message. The community may require that the sign owner cooperate with testing by programming the EMC for testing upon written notice. STEP 5 USE AN ILLUMINANCE METER TO MEASURE THE BRIGHTNESS OF THE EMC. Mount the sensor of your illuminance meter to a tripod and orient the sensor directly towards the face of the EMC at the measurement distance determined in Step 2. Ensure that the illuminance meter is set to measure footcandles up to two decimal places. As the display alternates between a solid white message and an “off” message, note the range of values on the illuminance meter. If the difference between the readings is less than 0.3 footcandles, then the brightness of the display is in compliance. If not, the display will need to be adjusted to a lower brightness level using the manufacturer’s recommended procedures. As the display alternates between a solid white message and an “off” message, note the range of values on the illuminance meter. If the difference between the readings is less than 0.3 footcandles, then the brightness of the display is in compliance. 10 © International Sign Association Item-3/73 HOW TO MEASURE THE NIGHT-TIME BRIGHTNESS OF AN EMC—WITHOUT CONTROL OF THE SIGN (Note: This method requires two individuals, but does not require operational control of the EMC.) ISA RESEARCH There will be instances where the EMC illumination needs to be evaluated to ensure that it does not exceed the brightness levels established in the municipal sign ordinance. If the municipality is unable to obtain access to the sign controls or attempting to take the measurement after business hours, this method should be followed. Unlike the six-step process described previously, this process measures the difference in brightness between the sign in operation and when the sign is completely blocked from the illuminance meter. This proce- dure is extremely simple and requires only an illuminance meter and a piece of painted cardboard cut to the proper size. STEP 4 POSITION THE TESTERS. Based on the size of the digital display, the person conducting the test should position themselves as close to directly in front of the digital display as practical, at the appropriate distance (calculated in Step 3). A helper should position themselves about 7 ft. to 10 ft. in front of the light meter and hold up an opaque, black sheet of material that is roughly 12 in. high by 40 in. wide. (Regular cardboard painted matte black works well for this.) The sheet should be positioned so it blocks all light from the EMC, but still allows the remaining ambient light to register on the illuminance meter. STEP 1 OBTAIN AN ILLUMINANCE METER. (See previous Step 1) STEP 2 DETERMINE SQUARE FOOTAGE. (See previous Step 2) STEP 3 DETERMINE THE MEASUREMENT DISTANCE. (See previous Step 3 or use (Area of Sign in Sq. Ft. x 100)) This helper should use a cardboard sheet to block the EMC light from the footcandle meter. This will establish the baseline footcandle reading. After the cardboard block is held in place, a reading should be taken for the ambient light. House Lights Commercial Lights Street Lights Landscape Lights Traffic Lights In this example, various light sources are impacting the photocell measuring 2.3 footcandles of ambient light. Traffic Lights Landscape Lights House Lights Commercial Lights Street Lights This is the baseline for the measurement. Write it down. © International Sign Association 11 Item-3/74 ISA RESEARCH STEP 5 USE AN ILLUMINANCE METER. House Lights Commercial Lights Street Lights Landscape Lights Traffic Lights The illuminance meter should be held at a height of about 5 ft. (which is approximately eye level) and aimed directly at the EMC. The illuminance meter will account for surrounding sources of light or the absence thereof. In this case our ambient light reading was 2.3 fc. The new light reading with the LED displaying a full white frame cannot read above 2.6 fc or 2.3 (ambient) + 0.3 (threshold). If a full white frame cannot be arranged, watch the meter to see if any ad exceeds 2.6 fc. At this point, readings should be taken from the illuminance meter to establish a baseline illumination level. (ISA recommends that the illuminance meter is capable of levels to 2 decimal places 0.00). Once the baseline level is established, add 0.3 footcandles to the baseline level to calculate the max brightness limit. (For example: Baseline reading is 3.15 footcandles. The max brightness level is 3.45 footcandles.) STEP 6 DETERMINE THE BRIGHTNESS LEVEL. Remove the opaque sheet from blocking the EMC. Watch the foot- candle meter for 3 to 5 minutes to see if the max brightness level is exceeded by any of the images on the sign. If the readings do not exceed the max brightness levels, then the EMC illumination is in compliance. If any of readings consistently exceed the max brightness level, the lighting level is not in compliance. In this scenario, the municipality will need to inform the sign owner of noncompliance and take appropriate steps to ensure that the EMC be adjusted to a lower brightness level using the manufacturer’s recommended procedures. If any of readings consistently exceed the max brightness level, the lighting level is not in compliance. 12 © International Sign Association Item-3/75 [Blank inside back cover] Item-3/76 1001 N. Fairfax Street, Suite 301 Alexandria, VA 22314 703.836.4012 Ph 703.836.8353 Fax www.signs.org Item-3/77 Pylon Sign 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R4-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 1 6of Item-3/78 A CUSTOMER APPROVAL SITE MAP APPROVED BY: ________________________ DATE: ____ / ____ / ____ Scale: 1/128" = 1'-0" © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R4-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 2 6of Item-3/79 ENGINEER FOR 115 MPH WIND LOAD (PER MUNICIPAL CODE) ± ” 8 / 5 0 1 - ’ 3 3 ” 3 - ’ 1 1 EXISTING PROPOSED ELEVATION A Scale: 1/8" = 1'-0" CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R4-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 3 6of Item-3/80 1 0” TOP VIEW 27’-11 1/8” 1 ’- 0 ” © 2022 Comet Signs Job Name: ENGINEER FOR 115 MPH WIND LOAD (PER MUNICIPAL CODE) ” 8 / 5 6 - ’ 4 1 SIDE VIEW END VIEW STRAIGHT ON FACE VIEW 19’-9” ” 1 1 - ’ 9 ” 5 - ’ 9 19’-6” SCOPE OF WORK: REMOVE EXISTING CABINETS FROM PYLON AND FABRICATE AND INSTALL NEW D/F FULL COLOR RGB EMC. • D/F TRIANGLE CABINET W/ TWO 15.85MM RGB EMC W/ PRE-FINISHED BLACK FILLER • ADD STUB PIPE TO RAISE SIGN UP AND REPAINT EXISTING POLE TO MATCH (P1) DETAILED SURVEY REQ. PRIOR TO FAB. NOTE: ENGINEERING REQ. TO FINALIZE MOUNTING Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R4-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 4 6of CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ A EMC PYLON SIGN QTY: ONE(1) Scale: 1/8" = 1'-0" ” 0 - ’ 0 5 ” 1 - ’ 0 4 Item-3/81 NORTH I-35 REAGAN & IN-N-OUT BURGER EXISTING SIGN PROPOSED SIGN I-35 NORTH BOUND Scale: 1/64" = 1'-0" CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R4-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 5 6of Item-3/82 EXISTING SIGN PROPOSED SIGN I-35 SOUTH BOUND Scale: 1/32" = 1'-0" CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R4-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. EXISTING PROPOSED TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 6 6of Item-3/83 Pylon Sign 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R5-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 1 6of Item-3/84 A CUSTOMER APPROVAL SITE MAP APPROVED BY: ________________________ DATE: ____ / ____ / ____ Scale: 1/128" = 1'-0" © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R5-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 2 6of Item-3/85 ENGINEER FOR 115 MPH WIND LOAD (PER MUNICIPAL CODE) ± ” 8 / 5 0 1 - ’ 3 3 ” 3 - ’ 1 1 EXISTING PROPOSED ELEVATION A Scale: 1/8" = 1'-0" CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R5-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 3 6of Item-3/86 1 0” TOP VIEW 20’-5 1/2” 1 ’- 0 ” ” 8 / 7 0 1 - ’ 0 1 SIDE VIEW END VIEW STRAIGHT ON FACE VIEW 14’-7” 14’-4” ” 9 - ’ 6 ” 3 - ’ 6 ” 9 - ’ 6 ” 0 - ’ 0 5 ” 3 - ’ 3 4 98.44 SQFT SCOPE OF WORK: REMOVE EXISTING CABINETS FROM PYLON AND FABRICATE AND INSTALL NEW D/F FULL COLOR RGB EMC. • D/F TRIANGLE CABINET W/ TWO 15.85MM RGB EMC W/ PRE-FINISHED BLACK FILLER • ADD STUB PIPE TO RAISE SIGN UP AND REPAINT EXISTING POLE TO MATCH (P1) DETAILED SURVEY REQ. PRIOR TO FAB. NOTE: ENGINEERING REQ. TO FINALIZE MOUNTING CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ A EMC PYLON SIGN QTY: ONE(1) Scale: 1/8" = 1'-0" © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R5-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 4 6of Item-3/87 NORTH I-35 REAGAN & IN-N-OUT BURGER EXISTING SIGN PROPOSED SIGN I-35 NORTH BOUND Scale: 1/64" = 1'-0" CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R5-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 5 6of Item-3/88 EXISTING SIGN PROPOSED SIGN I-35 SOUTH BOUND Scale: 1/32" = 1'-0" CUSTOMER APPROVAL APPROVED BY: ________________________ DATE: ____ / ____ / ____ © 2022 Comet Signs Job Name: Proposal Drawing Final Drawing Client: Client Name Location: 4507 N Interstate 35 Frontage Rd, Austin, TX 78722 Salesperson: Maggie Star Prj. Mngr.: Andy Leffler Date: 3/11/2022 Designer: Chris Brazell File Name: 66141-R5-Atty Tony Nguyen-Pylon.cdr Proposal #: 66141 Job #: n/a Revisions (M/D/Y)-(initials): desc. EXISTING PROPOSED TDLR #: 18010 MET #: E113766 1130 Rutherford, Suite 180 Austin, TX 78753 (512) 374-9300 T H I S D R AW I N G I S T H E E X C L U S I V E PROPERTY OF COMET SIGNS, LLC. ANY USE OR REPRODUCTION OF THIS DRAWING WITHOUT WRITTEN AUTHORIZATION FROM C O M E T S I G N S , L L C . I S S T R I C T LY PROHIBITED. OWNERSHIP AND COPYRIGHT IS RETAINED IN ACCORDANCE WITH U.S. AND INTERNATIONAL TRADEMARK / COPYRIGHT LAWS. 6 6of Item-3/89 BOA SIGN REVIEW COVERSHEET CASE: C16-2022-0004 BOA DATE: June 13th, 2022 ADDRESS: 4507 N IH 35 SVRD NB OWNER: Tony Nguyen COUNCIL DISTRICT: 9 AGENT: N/A ZONING: LO-NP (Upper Boggy Creek) LEGAL DESCRIPTION: LOT 1 TERRY ROSS ADDN VARIANCE REQUEST: exceed sign area of 107 square feet (maximum allowed) to 200 square feet (requested) and exceed sign height of 35 feet (maximum allowed) to 50 feet (requested) SUMMARY: provide signage for a multi-tenant office ISSUES: large trees over 35 feet high ZONING LAND USES LO-NP SF-3-NP Site North South LO-CO-NP East West SF-3-NP CS-CO-NP Limited Office Single-Family Limited Office; General Commercial Services Single-Family General Commercial Services NEIGHBORHOOD ORGANIZATIONS: Austin Independent School District Austin Lost and Found Pets Austin Neighborhoods Council Del Valle Community Coalition Delwood II Neighborhood Organization Friends of Austin Neighborhoods Homeless Neighborhood Association Mueller Community Association Neighborhood Empowerment Foundation Neighbors United for Progress Preservation Austin SELTexas Sierra Club, Austin Regional Group Upper Boggy Creek Neighborhood Planning Team Item-3/90 June 9, 2022 Tony Nguyen 4507 N IH 35 Svrd NB Austin TX, 78722 Re: C16-2022-0004 Dear Tony, Property Description: LOT 1 TERRY ROSS ADDN Austin Energy (AE) has reviewed your application for the above referenced property, requesting that the Board of Adjustment consider a variance(s) from LDC Sections 25-10-123 (B)(2)(a)(i) and (B)(3)(a) at 4507 N IH 35. Austin Energy cannot approve the variance request at this time, as more time is needed to research whether the proposed sign can meet required clearances from the existing overhead electric lines located adjacent to this site. A decision will be reached regarding this variance request prior to the July 2022 Board of Adjustments hearing. Please use this link to be advised of our clearance and safety requirements which are additional conditions of the above review action: https://library.municode.com/tx/austin/codes/utilities_criteria_manual?nodeId=S1AUENDECR_1 .10.0CLSARE If you require further information or have any questions regarding the above comments, please contact our office. Thank you for contacting Austin Energy. Cody Shook, Planner II Austin Energy Public Involvement | Real Estate Services 2500 Montopolis Drive Austin, TX 78741 Cody.Shook@austinenergy.com Item-3/91 T T E N N E B T R O P R A I N O M R A H T R O P O AIR B T 5 S N IH 3 T S 5 3 O 51 B T 5 N N IH 3 LY L E N N O C 46T H 35 5 3 O P T R 5 3 O A I R B T 5 S N I H 3 D O O ELW F AIR W O O D F E R N W O O D 45 T H N O S K R A L C B S D U 5 3 H I O T B S T R O P R A I B S D R V S 5 3 H I O T T R O P R A B S D U 5 3 H I O T T R O P R A I 44TH 43 R D R T O P I K C E R D E P P 5 U 3 K C E R D E P P 5 U 3 N IH 3 5 U B T O A IR D N C R E S T W O O D ! ! ± SUBJECT TRACT PENDING CASE ! ! ! ! ZONING BOUNDARY NOTIFICATIONS CASE#: LOCATION: C16-2022-0004 4507 N IH 35 SERVICE RD NB This product is for informational purposes and may not have been prepared for or be suitable for legal, engineering, or surveying purposes. It does not represent an on-the-ground survey and represents only the approximate relative location of property boundaries. 1 " = 250 ' This product has been produced by CTM for the sole purpose of geographic reference. No warranty is made by the City of Austin regarding specific accuracy or completeness. N O R W O O D B E N T W O O D O D O W P A R K Item-3/92 C16-2022-0004 12926252 0219120112 LO-NP (Upper Boggy Creek) Council District: 9 Item-3/93 Item-3/94 Item-3/95