Game Changer for Data Modernization! (Hint: It’s not Hadoop)

Sanjay Vyas

CEO, Diyotta

Recently, I had the privilege to co-present a webinar on data modernization with Mark McKinney (Director of Analytics at Sprint) and Piet Loubser (VP of Product and Solutions Marketing at Hortonworks). The theme across all the presenters was pretty loud and clear – if you have not started thinking about data modernization yet, it’s time. If you are already onboard then plan big, act small and move fast. In this blog, I expand on some of the key takeaways from the webinar.

Organizations are becoming data conscious and recognizing the potential of the data hence, the modernization journey, as Mark very well pointed out, should be closely aligned with business imperatives. We often make a common mistake by starting with technology imperatives like EDW Modernization and many other names but in my opinion, business outcomes should drive the technology strategy and not the other way around.

The game changer for data modernization is not the technology itself (and by no means I am trying to undermine the revolutionary and disruptive technologies out there), rather, it is the approach you take towards the modernization. Whether your modernization drivers include acquiring data that was never accessed before or improving the data management processes around ETL/ELT for optimization and/or reduce cost or centered around your organization’s cloud strategy, it’s the modernization approach which can become the game changer for your organization.

Mark mentioned during the webinar, plan your modernization strategy and when you are done, plan some more. Make sure that you have the buy-in from key business stakeholders and align the technology direction with the business outcomes. Take an iterative approach with iterations which result in incremental success. Realign if you find yourself on a wrong path which very well may happen as we tend get driven by rapidly evolving technology landscape.

Thinking big requires an ability to map the business priorities against technical direction, for e.g., hardware capacity is not just to manage the data volume only but also to ensure you have good compute power as well. Also look the hardware which is extensible in terms of adding more capacity in future rather than replacing the hardware. Also, some key considerations which may impact significantly down the line for e.g. real-time architecture. Lastly, business and IT adoption of the new strategy is another very important dimension in planning your strategy. Internal evangelization, using tools which can leverage existing skillset and building out a digital transformation strategy for better analytics are few of the key takeaways we learnt from Sprint’s journey.

As the telecom industry is going through the digital transformation phase, there are many battles which are yet to be conquered like Customer Journey Analysis, Network Forecast Analytics, Fraud Analytics, Call Center Analytics, Social Media Analytics just to name a few. All these business imperatives can trigger your modernization journey.

Putting the webinar link here for a webcast, I am sure it will be an interesting watch. Also, some additional references to a couple of telecom case studies. I would love to hear about your journey and any tips you have for the community who are in the same boat.

References:

Webcast link: http://bit.ly/2AoyveH

Hortonworks Diyotta Telco Case Study pdf: http://bit.ly/2kESDSJ

Network Forecast Analytics: http://bit.ly/2wSLTBZ

Hadoop Summit 2016: https://www.youtube.com/watch?v=mrC0osNORpQ


Webinar | SPRINT’S DATA MODERNIZATION JOURNEY

Webinar

About the webinar:


Sprint, being one of the largest telecom organizations in the US with over 60 million subscribers relies on analytics and insights derived from their vast data landscape to drive better customer experience, improved business operations and detecting fraud. Modernizing their data platforms turned into a key strategic undertaking with several complex parameters related to ingesting billions of records in a continuous cycle from hundreds of feeds coming from various operational systems, storing massive amounts of data in 100s of terabytes per day and provisioning the data in a timely manner to support several business analytics patterns.

A conventional approach with traditional technologies could not provide a comprehensive solution and would lead into a patchy and fragmented approach which is not scalable, suffers from several points of failure and creates maintenance challenges. Additionally, upgrading existing technologies would be a cost prohibitive endeavor.

In this webinar, we will hear from Mark McKinney, Director – Enterprise Data Analytics at Sprint about the business drivers, key success factors, and challenges faced while undertaking Sprint’s data modernization journey. You will hear how Sprint set about establishing a Hadoop data lake, ingested data from multiple environments, and overcame key skill shortages. You will also hear from Diyotta and Hortonworks about best practices for modernizing your data architecture to support transformational business initiatives.

WATCH THE WEBINAR

Speakers:


Mark McKinney
Director, Enterprise Data Analytics – Sprint
Sanjay Vyas
CEO & Co-Founder – Diyotta
Piet Loubser
VP, Product & Solutions Marketing – Hortonworks

View and download the slides:


Fill out the form below to download slides from the webinar.




By checking this box, Diyotta may use the information you have provided to keep you informed about products, services and offerings.


Modern Data Integration

Data Sheet


Massive amounts of new data are being created daily and there is a major paradigm shift taking place in the field of data and information management. No longer linear in nature, data that used to flow in one

direction is now moving fluidly in many directions. To further complicate things, the number of platforms and targets where data is being landed has also grown to include many new open source and purpose-built platforms. Given the change in the data landscape, traditional ETL architectures simply cannot manage the flow of data in this new paradigm.

Fill out the form below to download the data sheet:





How governed is your data lake?

Sanjay Vyas

CEO, Diyotta

Data modernization is inevitable – I seldom come across any organizations who are not investing in the modernization of their data platforms. There is a reason why the community stopped capitalizing big data as it became a regular routine term during technology discussions. Back in 2014, every conference was themed on big data however, gradually, the themes shifted to “cloud”, “AI”, “machine learning” which are the new buzzwords for today’s data landscape. With the ever-growing data universe around us, data modernization is no longer the fancy buzz word rather, it has become an essential part of organization’s overall strategy to stay ahead or stay competitive.

Gartner sums it up quite well as to why organizations need to modernize: “Existing data architectures are in most cases not ready for the future of data and analytics.” (reference: https://www.gartner.com/smarterwithgartner/2017-the-year-that-data-and-analytics-go-mainstream/)

For any new architecture, laying the right foundation and getting the priorities in order is utmost important and modernization is no different. While there are various technical and functional aspects to consider when developing a modernization strategy, data governance and technical metadata management are often overlooked.

 

Modern data landscape is already a beast of gargantuan proportions where we have more data points than ever before, hitting from every direction possible with several choices in terms of compute processes like Hive, Spark and diversity in terms of infrastructure whether it is across multiple regions, countries or clouds. Recognizing the data governance needs later in your journey and taking on the data governance challenges retrospectively can really become a costly undertaking prone to inaccuracies. It is important for the solutions experts to proactively manage the data governance priorities particularly around technical metadata for the modern data integration as that is the most difficult part to capture after the fact.

As someone said, the best way to learn something is from someone else’s experience – Register for the webinar to hear from @Alex Pain (Director of IT at Scotiabank), @Deepak Rangarao at IBM and myself on Nov 15th 11AM EST on how to build data governance practices in your data lake architecture. If you are not able to attend, please go ahead and register for a later viewing of the webinar at your convenience.


Diyotta and Clarity Insights Announce Implementation Software Services

Press Release

Clarity Insights joins Diyotta’s System Integrator Partner Program to Provide Implementation Services of Diyotta’s Modern Data Integration Suite

READ MORE


Webinar | Scotiabank's Governed Data Lake with IBM Information Governance Catalog

Webinar

About the webinar:


International banking spans across multiple countries where each country has its operational and analytical data platforms to fulfill business, ops and regulatory requirements. Enterprise Data Lake is a centralized Hadoop repository which acts as a crucial information asset for the entire organization however, there are multiple challenges in building an ecosystem which ensures that the platform can adhere to the enterprise standards in terms of security, governance and quality. Above all, this new architecture required a balance of tools and standards to make it a successful and continuously evolving strategy. In this session, we will talk about Scotiabank’s journey to establish a transparent global data lake using IBM IGC and Diyotta.

VIEW THE WEBINAR

Download the slides


Fill out the form below to download slides from the webinar.




By checking this box, Diyotta may use the information you have provided to keep you informed about products, services and offerings.


Diyotta Strengthens Advisory Board With Industry Veterans

Press Release

Diyotta announces the appointment of three new Board of Advisors for Marketing, Sales and Product respectively.

READ MORE


Modern Data Integration White Paper

Whitepaper

Advances in technology continue to accelerate the pace and competitive environment of business. Those organizations that are able to utilize information to analyze activities and trends, and create new insights are leading their industries. Business leaders rely on fact based decision-making and information analysis for their competitive advantage. Innovations in technology ranging from automation of manual activities to the interconnectivity of devices for the Internet of Things are contributing to the overwhelming amounts of data.

Fill out the form below to download the whitepaper:





Unified Big Data Integration White Paper

Whitepaper

In today’s increasingly fast-paced business environment, organizations need to use more specialized and fit-for-purpose applications to accelerate deployment of functionality and data availability to meet rapidly changing requirements and voluminous data growth. Download the whitepaper to learn about the latest innovations for integrating Big Data.

Fill out the form below to download the whitepaper:





Unified Data Integration In A Distributed Data Landscape

Whitepaper

Even in traditional environments we have seen complexity increase. Core online transaction processing (OLTP) systems have spread outside the firewall as companies adopt cloud-based packaged applications such as Salesforce.com and Workday. In addition, digitalisation has caused explosive growth in the rates at which session data and transactions need to be captured now that web, mobile and social commerce are all occurring. Also customer-facing applications provide a much richer user experience today, storing nontransaction data as well as transaction data.

Fill out the form below to download the whitepaper: