Three Challenges In Machine Learning Development and One Way to Overcome Them
Digital marketer turned technical PM.
One of the much-hyped topics surrounding digital transformation today is machine learning (ML). Admittedly, there’s more to it than just the buzz: ML is now, essentially, the main driver behind the artificial intelligence (AI) expansion with AI market set to grow up to over $5 billion by 2020.
With Google and Amazon investing billions of dollars in building ML development projects, we can rest assured the future depends on how successfully companies harness machine learning to add value to their businesses.
However, like any innovative technology, machine learning has its own bottlenecks and challenges that far not all companies today are able to overcome successfully to gain an innovation advantage over the competition and to transform digitally for better tomorrow.
Let’s take a look at top 3 challenges facing companies seeking to jump fast on the ML technology bandwagon and derive substantial business value from it, and let me show you on real-life examples how ML outsourcing can be a good way to overcome all 3 most critical challenges.
What is machine learning in a nutshell?
Simply put, machine learning is the ability of computer systems to educate themselves and make decisions without being specifically pre-programmed to do so. A close relative of computational statistics, it’s an integral part of a larger AI concept. ML is data-driven as it leverages big volumes of data to derive insights and make predictions.
On a day to day basis, machine learning already permeates our lives, although we may not be fully aware of it. Facebook uses ML algorithms to personalize our news feeds and to power its face recognition feature, Netflix uses it to offer us movies and shows that best match our taste, and our favorite e-store may use ML to predict the next product we are going to shop for and send us personalized offers. The weather forecasting software in our phones leverages advanced machine learning features to bring us exact weather forecasts every day, and Apple Siri is an example of what machine learning paired with natural language processing can do.
On a larger scale, ML is used in FinTech to derive insights on emerging trends, identify clients whose profiles look suspicious and prevent fraud. Today, ML literally saves lives, as it helps detect cancer cells a year before diagnosing cancer with conventional methods.
Industry giants are already reaping full-scale benefits: ML accounts for Netflix $1 billion savings in 2017, it’s also the force behind Amazon’s same day delivery option. Customers are getting used to it as well: Salesforce predicts that by 2020, 57% of business buyers will expect companies to already know what they need even before they ask for it.
Yet, although 80% of C-level execs believe AI boosts productivity and 42% of executives believe AI will be of critical importance within 2 years, many small and medium businesses are seriously lagging behind in terms of embracing machine learning. Moreover, even among enterprises with over 100,000 employees, only 50% currently have some sort of AI strategy, reports MIT Sloan Management Review.
Clearly, adopting ML involves many challenges. Among them are:
1) Lack of ML development resources
65% of technical recruiters claim talent shortage is the biggest hiring challenge faced when staffing IT and software development teams. The more complex the skills that are sought after — the more challenging it is to find and hire a relevant specialist.
Although many executives report having tweaked their tech job requirements to include proficiency in machine learning and data analysis, experts with these skills are still extremely rare.
According to Kaggle, only 4.5% of data scientists work specifically as machine learning engineers. Coaching internal talent appears to be an alternative, but educating in-house employees to the level necessary to successfully implement robust full-fledged ML projects would consume the other critical resource in a highly-competitive environment — the time. As of today, many established companies fear they would be outpaced by some newly emerged startups unhindered by neither legacy infrastructure nor software, and outsourcing their AI and ML-based project development to offshore or nearshore providers with access to untapped talent pools and vast internal expertise.
2) The high cost of ML talent
Today, the three most demanded skills on Monster are machine learning (ML), deep learningand natural language processing (NLP). Machine learning also tops the top 10 emerging jobs list on Linkedin. Being in high demand, AI and ML specialists request hefty payment packages: $300,000 to $500,000 annually in salaries and company stock — a worth of Rolls-Royce Ghost Series, according to the New York Times. Moreover, the demand for these experts is growing almost exponentially.
In contrast, in countries like Ukraine, a mid-level Python or Scala developer with the upper-intermediate or advanced English language proficiency commands an annual salary of around $61,000 (including all taxes, social benefits and a provider’s service fee).
The set of ML Engineer skills includes data structures, algorithms, computer architecture, computability and complexity, and the ability to carry out quality data modeling and evaluation. The skills are multi-faceted and diverse. Obviously, years will pass before graduates with solid knowledge of machine learning enter the job market and qualify to work on business critical projects. As a result, only 20% of executives feel their data science teams are ready for AI, while 19% have no data science team at all.
3) Long time to hire a high quality ML developer
As far as IT goes, the average time to hire a professional software developer resource with mainstream tech skills in the USA and Canada makes 60 days. Based on my firsthand experience working in the Ukrainian IT outsourcing industry, the average time to hire a Python developer (Python is one of the most commonly used programming language when it comes to ML implementation) is around 20 days which is 3x shorter than in North America.
Given the complexities involved, executives in small and mid-sized companies may be tempted to give up on the idea of developing their own ML solutions, since the total cost of ML resources can be overwhelming. However, these obstacles may be easily removed by outsourcing ML development and implementation projects to a specialist development provider in a lower-cost country. Similarly to AI outsourcing in general, offshore ML development companies have access to international talent pools, which helps eliminate the need to hunt for expensive local talent.
It takes a professional team of software engineers to build business standard ML solutions. Partnering with a trusted AI outsourcing company will help you build it for a reasonable cost. A reliable ML outsourcing provider will offer you a range of collaboration options: from extending your existing AI/ML development team with a number of “plug-and-go” resources to full-scale ML development project outsourcing.
Other than the pricing factor, the key indicators you should consider outsourcing your machine learning projects relate to the value ML will add to your business. Whether you need to establish the data collection process or derive insights from wide arrays of data you already have at your disposal, or you’re looking for opportunities to expand your business, or you want to automate a part of your business which is not tightly interconnected with your core operations, ML outsourcing is a way to go. Your in-house staff can focus on core product development, support and maintenance tasks, which are more business critical, while your ML outstaffing or outsourcing provider will get your the right resources for project development fast and cost-effectively.
Machine Learning Outsourcing comes to the forefront
Companies that want to stay ahead of the curve realize the importance of being flexible when it comes to making decisions about digital transformation or improvement of software delivery to end users, and the ability to seek talent and expertise beyond any limitations on the local jobs market is a hallmark of strategic thinking and business maturity. Why follow local peers and/or competitors that struggle to find, hire and, what’s even more important, retain the experienced and highly competent ML/AI workforce, while you distribute your ML development team among a few remote destinations and hire managers to oversee and be responsible for project development and deliveries on-site? You win from cost arbitrage, faster time to hire and fire (it’s not that easy to fire a bad developer in the UK or USA, while in countries like Ukraine you can get rid of your slow achieving programmer with just a 2-week notice and no admin hassle), a provider’s experience and expertise, its ability to supply and retain your ML dev talent, and eliminate overheads related to your staff attrition, reskilling or upskilling, and so on.
Today, the world’s top outsourcing destinations for ML projects are located in Eastern Europe with over 3,000 machine learning specialists based in Ukraine, according to LinkedIn alone. Global tycoons like Amazon, Cisco, Google, Intel, Nokia and Microsoft are opening regional offices in countries like Poland, Ukraine, Belarus and Romania. Clearly, they have realised the role that offshore and nearshore outsourcing plays in delivering high quality solutions and reducing time-to-market, which is essential for modern businesses to stay afloat.
For instance, one of the UK’s leading media content providers that had successfully launched a platform similar to 9gag and Reddit, was looking to enhance its platform monetization by increasing the end-user’s loyalty and willingness to click and go to the partners’ offers. The only way to achieve this was to provide content which would be interesting, relevant and eye-catching for the end user and could keep them engaged for a while. The company was seeking ways to extend technological man- and brain-power to boost the platform’s growth and achieve global market expansion, but it faced serious challenges in the local tech market: they used both in-house recruiters and a staffing agency to find and hire ML talent. Time to find good candidates and hold interviews in-house was very long and working with the agency proved to be expensive as they charged a service fee of up to 25% of each team employee’s annual salary.
Having spent a month doing it within the UK, the company revisited its strategy and turned to us for solution development. We helped them hire a team of 2 extra developers (skilled in Python, Scala and Apache Spark) in Ukraine within less than 30 days, on-boarded them within a week and had an up-and-running ML development team working on the client’s project within less than 2 months. To retain 100% PM control, the company assigned a site manager to also act as their remote tech lead who was able to visit the team in Ukraine on a monthly basis and communicated with them on a daily basis via Scrum and virtual tools.
As a result of ML outsourcing, the UK media content provider managed to increase its audience monetization crucially (for the first six months after the deployment, click-and-go for partner offers grew by 20% compared to the previous period), whilst the overall audience increased by approximately 40%.
The company was able to reach the breakeven point in five months from the offshore project kickoff.
All in all, businesses are increasingly looking for opportunities to harness the power of data science and machine learning to enhance customer services, streamline their operations and gain insights from large data sets. Given the scarcity and the cost of talent, ML outsourcing is a reasonable alternative to building in-house teams from scratch and educating existing employees.