About VIA SMS Group
Founded in 2008, VIA SMS Group developed an entirely new approach to issuing quick loans using predictive analytics and machine learning. VIA SMS Group uses advanced algorithms to assess whether an applicant is fraudulent and, if not, what his or her credit profile looks like in order to determine whether or not to underwrite a loan.
VIA SMS Group writes the decision algorithms in the R programming language. Before using ScienceOps, VIA SMS had to rewrite these algorithms from R to the server-side language of PHP in order to deploy models into their web and mobile apps.
“There were several problems with getting the modern data analytics models to work in production. First of all, we were basically limited to linear or logistic regression models and decision trees since anything we wrote had to be implemented in PHP. Models with many interaction terms were entirely out of scope,” explains Dmitrijs Ļvovs, Risk Manager at VIA SMS Group.
“We were basically limited to linear or logistic regression models and decision trees since anything we wrote had to be implemented in PHP.”
“It also took a lot of time to make even minor modifications like adding new predictors because we were handing off models to the dev team. Updates could easily take a few days per model since they were dependent on coordination between teams and manual input. Rewrites were prone to typos and syntactical errors, since the languages handle data types differently.”
VIA SMS Group wanted to utilize the most cutting edge predictive modeling techniques in their production systems. They also wanted to minimize the use of IT resources needed to add new data to their scoring models.
At first, VIA SMS Group planned on tackling these goals by building their own production system.
“We initially intended to go the route of building an internal solution that relied on R installed on a server to slurp up models in R and link to our web systems via some shell scripts.
“We intiailly intended to go the route of building an internal solution...”
Build vs. buy
“Building infrastructure that can deploy, test and monitor models is a really big undertaking. It would have taken us several dedicated engineers to build it, plus ongoing maintenance and upkeep. We realized that it’s not our data science team’s job or expertise to build reliable services, and it’s not our dev team’s role to build software to supports advanced statistical and machine learning algorithms.
“Ultimately, even if we had built our own production system that relied on R, we wouldn’t have actually reduced the constraints on the dev team, since our data science team knows R, but our dev team doesn’t. Building an in-house solution just didn’t make sense. It was going to be an inefficient use of valuable resources and it wouldn’t have eliminated the original issues we encountered when porting over models,” explains Dmitrijs.
“Building an in-house solution just didn’t make sense. It was going to be an inefficient use of valuable resources and it wouldn’t have elminated the original issues we encountered when porting over models.“
Making a decision
Rather than build an in-house system, VIA SMS Group turned to ScienceOps for an efficient and reliable solution.
“We are extraordinarily cautious about the stability of any new solution, especially those coming from startups. Having used ScienceOps for months and never experiencing a bug or outage, it’s clear to us that Yhat has put a tremendous amount of work into ensuring that the product just works. Any time we’ve needed technical support it’s been excellent—we’ve always gotten answers in a day’s time, despite operating in very different time zones.
Implementing a solution
“With ScienceOps, we have a dedicated platform for deploying models into production without retraining our data science team or incurring massive technical overhead.
“With ScienceOps, we have a dedicated platform for deploying models into production without retraining our data science team or incurring massive technical overhead.”
VIA SMS Group was able to implement ScienceOps and begin to reap the benefits of the platform almost instantaneously.
“It took us less than a week to test and deploy our initial model. After we’d integrated that model into our systems, we were able to deploy and embed all of our other models into the production system in a single day.
“It took us less than a week to test and deploy our intial model. After we’d integrated that model into our systems, we were able to deploy and embed all of our other models into the production system in a single day.”
Measuring the results
“We no longer have to worry about the risk of error inherent in rewriting models because we can deploy our models in the native language they were trained and tested in. ScienceOps eliminates the headache related to integrating R’s extremely rich libraries with our live systems. R packages like randomForest are now extremely easy to embed into production models for real-time use.
“With ScienceOps, our data science and risk analytics teams can test, deploy and monitor models in real time, so the feedback loop for retraining and fine tuning models is significantly faster. On top of that, because the data extraction and manipulation code in the model training environment is identical to our production code, we’ve cut out an enormous chunk of testing that was previously necessary.
For VIA SMS Group, deploying models via ScienceOps has been a definitive and measurable success.
“Model retraining used to take us four weeks. Now it takes just a couple of hours for all group subsidiaries. Since our models have gone live, we’ve reduced the number of manual application reviews by five percentage points. By using the advanced modeling techniques made available natively with ScienceOps, we’ve dramatically improved our model performance as measured by a 13% lift in ROC curve. For VIA SMS Group, deploying models via ScienceOps has been a definitive and measurable success.
How to learn more
If VIA SMS Group's painful experience with deploying models into production apps sounded all too familiar, I'd highly recommend starting a conversation with the team here at Yhat.
Our team of dedicated sales engineers would love to talk with you about your use case and discuss how ScienceOps boosts model performance and drastically reduces engineering time and costs.