In today's unforgiving global economy, those organizations that compete on analytics stand the best chance of outsmarting the competition. The only catch is, they need skilled professionals who know how to manage, mine and draw actionable insights from all the "Big Data" now streaming across enterprises.
Many organizations already have long had database administrators and analysts, but a new breed of skill is required not only to manage and mine Big Data in huge volumes (now into the petabytes), but also address a rising amount of data in "unstructured" formats, such as video, graphics, weblog data and documents. These professionals also need to be able to deliver analytics in or close to real-time to decision makers.
These professionals will be asked to look at information in new ways that may even run against the grain of traditional processes within their organizations. Skillwise, they may be tasked with implementing the open-source Hadoop framework, considered one of the most effective tools for taking mountains of Big Data and coverting them into more compact files that are digestable by existing applications.
Even more important, however, is they will help guide decision-makers to ask questions they never could have asked before. Once there are ways to crack open the Big Data puzzle, nothing is off limits. Consider, for example, as reported in The Wall Street Journal, how Xerox dramatically reduced turnover in its call centers by looking at information about job applicants in new ways.
ComputerWorld's Tam Harbert recently explored the skills and needs organizations are searching for in the quest to manage the Big Data challenge, and also identified five job titles emerging in the Big Data world. Along with Harbert's findings, here are 7 new types of jobs being created by Big Data:
- Data scientists: This emerging role is taking the lead in processing raw data and determining what types of analysis would deliver the best results. Typical backgrounds, as cited by Harbert, include math and statistics, as well as artificial intelligence and natural language processing.
- Data architects: Organizations managing Big Data need professionals who will be able to build a data model, and plan out a roadmap of how and when various data sources and analytical tools will come online, and how they will all fit together.
- Data visualizers: These days, a lot of decision-makers rely on information that is presented to them in a highly visual format -- either on dashboards with colorful alerts and "dials," or in quick-to-understand charts and graphs. Organizations need professionals who can "harness the data and put it in context, in layman's language, exploring what the data means and how it will impact the company," says Harbert.
- Data change agents: Every forward-thinking organization needs "change agents" -- usually an informal role -- who can evangelize and marshal the necessary resources for new innovation and ways of doing business. Harbert predicts that "data change agents" may be more of a formal job title in the years to come, driving "changes in internal operations and processes based on data analytics." They need to be good communicators, and a Six Sigma background -- meaning they know how to apply statistics to improve quality on a continuous basis -- also helps.
- Data engineer/operators: These are the people that make the Big Data infrastructure hum on a day-to-day basis. "They develop the architecture that helps analyze and supply data in the way the business needs, and make sure systems are performing smoothly," says Harbert.
- Data stewards: Not mentioned in Harbert's list, but essential to any analytics-driven organization, is the emerging role of data steward. Every bit and byte of data across the enterprise should be owned by someone -- ideally, a line of business. Data stewards ensure that data sources are properly accounted for, and may also maintain a centralized repository as part of a Master Data Management approach, in which there is one "gold copy" of enterprise data to be referenced.
- Data virtualization/cloud specialists: Databases themselves are no longer as unique as they use to be. What matters now is the ability to build and maintain a virtualized data service layer that can draw data from any source and make it available across organizations in a consistent, easy-to-access manner. Sometimes, this is called "Database-as-a-Service." No matter what it's called, organizations need professionals that can also build and support these virtualized layers or clouds.
Here are some examples of actual recent job listings, pulled from online job boards, which even include "Big Data": in their job titles. Note how these jobs not only require technical proficiency, but also an ability to communicate and solve business problems:
- Big Data Architect – Analytics (major retailer): "Focused on creating views on top of structured and non-structured data and presenting that data in a portal framework. Will initially focus on data mining and data visualization using the latest in open source data mining/data presentation technology.... In addition, the team will begin to pull in other sources of data such as BI, user feedback and social to help us better understand our customer."
- Big Data Analyst (scientific instruments manufacturer): "Help better understand, test and use vast volumes of data. Support the business through advanced analysis and design, maintenance, and implementation of reports and databases. Design and build scalable infrastructure and platforms to collect and process very large amounts of structured, unstructured and real-time data. Analyze large volumes of data from disparate types of sources and present findings to senior management."
- Principal Engineer, Big Data (mobile telecom provider): "Skills will be applied to solving problems impacting millions of customers. Explores large data volumes using state of the art tools and techniques to find solutions to practical business problems."