The largest explosion in history is the Big Bang. Within an almost impossibly small fraction of a second, the universe bloomed, growing faster than the speed of light, doubling in size time and time again, expanding from an infinitely hot and dense horizon – from the size of a single atom to about the size of a small orange. This momentary phase is called cosmic inflation, and theoretically, this set in motion all the cosmic forces that shape matter, space, and existence.
Today, the sheer amount of stuff being digitized and recorded is increasing at an unprecedented rate. Likewise, business processes are evolving to become more digital, and to include informational assets such as photos, recordings, content, and contributions from vastly distributed sources. Technologists are therefore in the middle of the digital big bang, and the associated inflationary forces at work are beginning to heat up.
Richer data means denser data
On the cusp of the bang, not only is the amount of data expanding faster than at any other point in the history of computing, but the rules are changing because of it. File sizes are growing exponentially. Flow volumes combined with the snowballing size of bigger aggregate data are a new condition of executing digital business processes.
means rich data. But like a heavy-cream pound cake, things that are rich also tend to be dense. Denser data is harder to handle given the limited carrying capacity of outdated legacy infrastructures or multiple single-point solutions. The increase in data density makes moving data into storage systems like Hadoop challenging. A large file in motion can stall productivity, and the chance for transfer error is magnified as the file size grows. Because of data density, getting large data sets into analytical platforms by relying on legacy systems is a near-impossible feat.
2. The Imaginary Road Trip
Consider a lengthy journey in a car. Not your everyday commute; but more the marathon-style drive long-haul truckers do for a living. Let’s say you start in Chicago and aim for California. Determine the optimal path to your goal. The fastest route generally wins out. And by condensing the transit duration it means:
• Limiting the time cooped up in your car
• Less anxiety because you’ll be using established, well-maintained roads
• Fewer rest stops and fewer meltdowns from young backseat passengers
• More productivity and time spent enjoying the destination
Now let’s imagine two versions of the trip.
You would hop in your 1931 Ford Model A and get on the 2,448-mile stretch of road called Route 66. Even though the 40 HP clunker was pretty reliable for its day, you would still require constant attention to avoid a breakdown, and hawk-like vigilance to avoid the deadly winding hazards that could cut your trip short.
To Be Continued…