Tech Corner
Big Data Provides Unique Challenges & Opportunities By Reagan Waskom, PhD
Big promises are being made for big data. The tsunami of data resulting from new technologies acquiring observations rapidly and inexpensively has created headaches, as well as intriguing opportunities.
Satellite images, wireless sensor networks, web-based photographs, and text, audio, and model output all produce data that must be stored, processed and analyzed to create useful and reliable information. Data in and of itself does not enhance our understanding or management decisions; it must be transformed into information that is accurate, timely and reliable to become truly useful.
Data acquisition capacity has grown to the extent that a new branch of information sciences has emerged, known as “big data.” Big data has been called a fad in scientific research, but perhaps it is more accurate to call it a “hot topic,” knowing that the cascade of data from ever-expanding new technologies will only continue. Numerous scientific conferences and papers on the topic of big data have occurred since the Obama Administration launched the Big Data Research and Development Initiative in 2012 to “greatly improve tools and techniques needed to access, organize and glean discoveries from huge volumes of digital data.”
Big data has been described as high- volume, high-velocity and high-variety information in excess of 1 terabyte, which is too large for a single machine to handle. Traditional techniques are insufficient in analyzing this amount of information.
This definition is fluid and may soon be described in petabytes, but it also includes the velocity at which the data is acquired from multiple independent data sources. Thus, cloud-linked servers are typically needed to adequately store and process the data. Real-time acquisition and processing that enable trend and pattern detection to provide actionable insight, improved decision-making and competitive advantage are the goals of businesses and government agencies seeking to exploit big data. In other cases, the goal may be to enable public access to useful, interesting or important information.
A number of questions must be resolved while developing new data technologies and capacity. For example, who owns big data when it is crowd-sourced or provided by multiple public agencies and private entities? How does the information remain secure and individual privacy protected? From a scientific point of view, what about data quality and veracity? How can sampling bias and misinterpretation be avoided? Again, data itself is not the goal — it is the information gleaned from the data that can enhance understanding of trends, processes, demographics, etc.
Ratepayer data aggregated from multiple public water systems might be an example of using big data to determine statistical patterns that suggest significant correlations and trends in water use and conservation, to forecast future demands
and ultimately to optimize coordination of resources. Water managers with multiple sources of water supply could also benefit from better data-driven forecasting and real‐time operations decision support. Sensor technologies have arrived on the market to help water utilities survey underground pipes and detect present or potential leaks. Smart meters could help both managers and individual users fine- tune their systems.
In terms of academic research, both the National Science Foundation-funded National Ecological Observatory Network and the Consortium of Universities for the Advancement of Hydrologic Sciences, Inc. networks have been organized to provide big and open data to researchers. NEON represents the largest single investment in ecological research data ever made. This research infrastructure is transforming the ability to advance data visualization and statistical methods to understand patterns, and trends processes and detect outliers.
The value of big data is the opportunity to answer big questions. What is also exciting about big data and open data is the potential for innovations that can substantially improve decision-making capacity. As the data tsunami keeps coming, the power of that data to help solve big water and natural resource challenges is ours to capture.
Reagan Waskom, PhD, is the director of the Colorado Water Institute at Colorado
State University, Fort Collins, Colorado. Waskom is a member of the department of soil and crop sciences at CSU, where he has worked on various water-related research and outreach programs for the past 27 years, conducting statewide educational and applied research programs on water quality, water quantity, water policy and natural resource issues related to water use.
irrigationtoday.org 25
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44
