What is Big Data?

Big Data was created or if not created at least taken mainstream by the powerhouse known as Google.  So, who better to turn to ask what the definition of Big Data is:

big da·ta

noun

COMPUTING

1. extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.

   "much IT investment is going towards managing and maintaining big data"

I would say this is a fairly accurate representation of what it means to me with the caveat replacing "especially relating to human behavior and interactions" will soon be replaced with more industrial terms, like machines, robots, software or business processes.  

Big Data processes like FarmStreams can constantly be verifying conditions in barns and alerting when conditions are not ideal for maximum production.  In addition, additional "smart automation" can be installed where several dependents are monitored and when conditions are met then equipment is remotely controlled to perform different tasks.  

Some good examples of a simple smart command would be to automate a bin slide to switch bins when one runs empty, open the full bin and shut the empty bin and order new feed for the empty bin.  That is a simple automation that could be accomplished pretty easily with Big Data.

A more complex automation that could be done is to calculate weight of feed consumed per bird and once it exceeds a certain percentage of body weight for the day to switch feed rations or stop feeding for the day.  Additional complexity could be added to cross reference water consumption before making a decision.  I am not sure if this is a realistic scenario but it is illustrating that many different streams can come together to automate a variety of decisions inside a barn.

Furthermore, within animal agriculture this level of data collection from the farm has never been available.  Most data that is available remains on the site.  Once the smart professionals within the animal ag industry get their hands on this amount of production data I am certain the value they find and impact will be tremendous.

So, how big is our data?  The data platform, Grovestreams follows the Google Big Data architecture with its own wrinkles.  This is called Hadoop.

So, what can we do?

• Farmstreams can store 94,000,000 samples for each stream - this is 3 years of one-second resolution.
• 140 different statistics for each sample available for each rollup period from one second to many years, with customizable time definitions in between.
• Each sample can have unlimited derivations (formulas) occurring with them. 
• The derived samples can also have unlimited derivations (formulas) occurring with them.
• Near real-time processing and flow-through.
• Data in / out seamlessly from cloud data platform back to the enterprise level system.  Users can add us as another user interface; but don't have to.

Most livestock farms will not need single second resolution on their data; but it is nice to know the data platform was built to handle that heavy of a workload.  The ability to run derivations / formulas in real time converts several data feeds into valuable information that ensures the livestock barn is optimal.

That is what Big Data means to us.  We are focused on data quality from the sensors.  Connecting the sensors in rugged, tough environments.  The data piece on the back-end.  Optimizing the production environment for the best performance possible.