Canary in the Data Mine

I spent a couple of weeks last month  interviewing experts about the big data workforce—that mix of IT pros, statisticians, mathematicians, engineers and functional business experts who companies need to tame their burgeoning databases. The resulting story, Building Your Big Data Team, is posted at Data Informed, a new website focused on big data technology, applications and management. In my story, there’s some advice for IT and business leaders about education and training for data scientists.

Meanwhile,  CIO.com has posted my feature, Big Data in the Real World Isn’t So Easy, about the challenges facing companies that want to analyze massive amounts of data. CIOs I interviewed for that story also talked about how they were rethinking the way they staff their data and analytics teams. One big focus: deepen connections between data analysts in IT and business domain experts.

As always, when confronted with new technology, IT leaders are figuring out what to do. But what’s happening now is triage: recruit talent from other companies, send developers to learn new tools, identify employees who can deepen their mathematical, engineering or domain expertise on the job.

Today’s demand for data experts is only the beginning. Last week, the Conference Board’s Gad Levanon and Jessica Forde blogged about new Bureau of Labor Statistics Data predicting the fastest growing occupations in the next decade:  “The rate of increase in market research analysts, personal financial advisors, and financial analysts indicate the growing need for professionals who can use data to aid in decision making processes.”

I wonder if our education system can fill the pipeline fast enough.

Big Data and the Future Workforce

I have spent the past couple of months working on projects that turn out to be related: a feature on Big Data analytics, and one about motivating students to pursue STEM–science, technology, engineering and mathematics–careers. (Both projects are in progress–publishing sometime this winter, I expect.)

For the analytics feature, I interviewed an advertising agency CIO. Here’s what he said when I asked him about what the impact of new investments in Big Data technologies and analytics platforms had been on his staff: “There’s such competition for anyone who understands the media business and has a math background.”

That doesn’t sound like big news. CIOs are always looking for IT professionals with business knowledge, and it shouldn’t be a surprise that increasing demand for analytics would generate a need for more technologists who are good at statistics. But the kicker was what he said next. It’s not just IT that needs business-savvy math experts. The business units need them too.

By coincidence, a few days later I was visiting my daughter’s grade-school classroom, where the kids were doing an exercise analyzing “mystery” data (a set of unidentified things that had varying lengths or heights) and generating hypotheses about what the data might be describing. Teaching students, even the really young ones, how to understand data and statistics has been a part of our local curriculum for at least a few years now. But I don’t think kids hear enough, early enough, about what they’re going to be able to do with these skills. Nor do they always get the opportunity to apply them in ways that prepare them for college or the workplace.

I think this probably as true now as it was when I was in high school, and a teacher who knew I wanted to be a writer told me not to worry about Algebra. He reinforced my teenage indifference about the subject. Fortunately, I got another chance to appreciate math, via a graduate school statistics course, before I ended up in roles that required me to conceive, write and edit scores of articles based on survey data.

A math teacher who I interviewed for the story about STEM careers told me that he didn’t always have a good answer for students who ask why they need to learn, say, trigonometry. “There’s no such thing as a math career unless you’re going into academia,” he says. He’s made an effort to learn about how math is applied in the workplace, and he concludes that students shouldn’t learn the subject in a vacuum, as a set of concepts and procedures disconnected from real-world problems that professionals have to solve. Other experts I talked to echoed this teacher’s assessment, not just about math but also about other STEM subjects.

It makes sense that if you want to turn students on to careers in science and technology, you have to show them what the work is like and inspire them with examples of challenges they could tackle as professionals or skilled technicians. For that matter, you have to show them how science and technology may permeate careers that aren’t, on the surface, related (such as sales, marketing or public relations for a technology, healthcare or engineering company). The world is interdisciplinary.

Maybe kids who graduate from high school shouldn’t just have to pass standardized tests to demonstrate they’re knowledgeable in core subjects. Maybe they should also have to show they know what to do with that knowledge through projects that simulate how adults work together and the kind of work we do. I don’t know how you test for that, or if you should. Maybe you have high school seniors do a special project as a graduation requirement as well as incorporate problem solving experiences into regular coursework. This could make kids’ education more rigorous, more practical and probably more interesting at the same time. Whether they go to college or get technical training, they’ll be better prepared to go to work and more attractive as job candidates.

At least I think so. Makes me wonder what, if CIOs and others who hire people for STEM positions could reshape what students are taught, they’d want to see.