The cloud – just another tool set

February 23rd, 2017 by Stephen Jones Leave a reply »

The cloud is a term that has for long time been the subject full of hype.

Conferences, blogs, media outlets tell us the cloud is the answer, the cloud is cheaper, the cloud is the way of the future, the cloud handles your DR, the cloud manages BI, IoT, etc. Major vendors, none more than Microsoft, are pushing the message of SMAC : “social first”, “cloud-first” “analysis fist’ and “mobile first”.

For many of us reactions vary from confused, to concerned, or even angry. There are still many who dismiss the idea of ‘cloud anything’ when it comes to data.

The reality is that almost everyone uses the cloud daily, Google, Hotmail, GPS, Facebook, You tube etc.
The cloud is a tool, the can really enable you to solve issues, some unsolvable any other way, without getting caught up in the details of implementing every little part of the system. That’s a any idea most of us embrace, even if we never thought about it. How many of us deal with hardware? How many of you install or configure Windows? Who day to day worries about SQL backups, or has scripts/tools/products to auto implement the back up new databases?

The reality is that few of us have ever seen an email server, or an erp a production database server with their own eyes, despite connecting to many.

We all move at different paces. Some of us still deal with SQL Server 2008, 2005, 2000,. Some of us need to manage those platforms for years to come. Using one application on the cloud does mean now work for IT on premise. Helping to build applications on Azure SQL Database and deal with data integrity, quality, and security issues, through a remote connection. Device connectivity and internet connections, and printers still need managing.

The cloud gives us a new set of tools and services that take away some of the details and drudgery. Think electric saws and power drills instead of hand tools. Sometimes that’s fantastic, and it enables more rapid, more scalable deployment of resources. Sometimes it’s dangerous because the vendors haven’t completely thought through the process, or integration, or latency, or licensing and contractual term.

We shouldn’t be doing many tasks that can be easily automated and done better, faster, cheaper.
When we don’t need to manage day to day management of hardware, databases, and interfaces we can look upwards and outwards and consider to use the tools, to ensure our organizations continue to improve effectiveness and efficiency.

So what tools does Microsoft offer on azure?

Cortana Intelligence Suite is an example – Microsoft’s big data, machine learning, and analytics platform.
Whether you work with a company that buys “best of breed” solutions and you manually manage your data or you only have one system but you lack the ability or wherewithal to analyze it, CIS can help.

CIS has several components and each has an intended purpose, though you can do similar functions across several elements, and may only use a few of those for your business.

Microsoft Azure
Azure’s intent is to provide cloud hosted virtual servers, apps, and services to Microsoft’s customers.
Azure offers developers both Microsoft and third party tools to operate on the cloud and takes full advantage of the IoT. Microsoft is playing to win the internet. It already has more data centres than Google and Amazon combined. The rapid improvements of the platform are steadily winning over developers and IT professionals.

Azure Data Catalog
Use it to tie together different types of data together from many sources, when all the fields or tables may be labeled differently. The Azure Data Catalog allows a user the ability to look up a data source in a common place in the cloud. The future of data management will tie together those end users with business knowledge and allow them to identify data relevant to your IT team.

Azure Data Factory
Azure Data Factory provides tools to pull together all your data into one place, to transform it, and to distribute it! (Pelluru, 2016)

Azure Event Hub
The Azure event hub is used for truly Big Data and the IoT. It allows mass data to be brought in for analysis. and is the opening to funnel in large amounts of data, like: temperature monitoring on all your reactors or the status of sensitive product shipments. Consume the data e.g. for behavior analysis on a mobile device.

Azure Data Lake
If Azure Event Hub is the front door, then lake is the storage closet for large amounts of data.
Use the lake to store all your data in its original non-relatable format, and integrate with any data warehouse you already have. It differs from Blob storage solely mainly in the amount of data you’re collecting.

Azure SQL Database
The SQL database needs no introduction, but how do SQL databases relate to Azure? A SQL database is where your data resides in tables, with key identifiers, which makes it relatable to other sets of data. SQL is simply the coding language for this particular database. The Azure SQL database puts it in the cloud.
But I don’t want my data in the cloud! That’s not secure.” Really ? Do you even bother to look the door of your server room? Are you really up to date with all he security patches and hotfixes?. I challenge you to rethink your philosophy and do a little research on the matter. Storage in the cloud is generally more secure than an on-premises solution, depending on several factors.

Azure Machine Learning
AML is where you go to create your machine learning models and API’s. What is machine learning?: It’s an evolutionary self-learning and adapting process. A machine looks at the data and suggests something. If that something doesn’t make it better, then it will try something else. If that still doesn’t make it better, it will try something else. It will keep trying until it finds something that works. And then try to make it work better.
Deep Blue the IBM computer than won a chess match with Kasparov not only worked with brute force calculation but also developed its own heurstics to better evaluate which positions t analyse further.
Machine learning is also about predictive analysis based on data you feed the machine. One example is when you go to Amazon, or Google or Booking .com based on your past purchases/searches, they recommend items. That’s machine learning!

Correlation of data to establish possible relationships has many applications from advertising to analysing medical symptoms. How might one use machine learning in manufacturing? You could increase the yields of your finished goods when you have data based on a specific factor, what was done prior to the production (cleaning, maintenance, etc), which operator does the best job on a particular product, and even which raw materials to use. Based on how much data you put in on your raw materials, such as quality data, ML could help with product engineering.

The possibilities are nearly endless.
Azure HDInsight
HDInsight is Hadoop as a service. Hadoop is faster than moving those large files across your network. Hadoop is an open source, Java-based programming framework that supports the processing and storage of extremely large data sets in a distributed computing environment. It is part of the Apache project sponsored by the Apache Software Foundation. It allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.

As the World Wide Web grew in the late 1900s and early 2000s, search engines and indexes were created to help locate relevant information amid the text-based content. In the early years, search results were returned by humans. But as the web grew from dozens to millions of pages, automation was needed. Web crawlers were created. An open-source web search engine called Nutch evolved – the brainchild of Doug Cutting and Mike Cafarella who wanted to return web search results faster by distributing data and calculations across different computers so multiple tasks could be accomplished simultaneously. During this time, another search engine project “Google” was in progress based on the same concept – storing and processing data in a distributed, automated way so that relevant web search results could be returned faster.

In 2006, Cutting joined Yahoo and took with him the Nutch project as well as ideas based on Google’s early work with automating distributed data storage and processing. The Nutch project was divided – the web crawler portion remained as Nutch and the distributed computing and processing portion became Hadoop (named after Cutting’s son’s toy elephant). In 2008, Yahoo released Hadoop as an open-source project. Today, Hadoop’s framework and ecosystem of technologies are managed and maintained by the non-profit Apache Software Foundation (ASF), a global community of software developers and contributors.

Azure Steam Analytics (ASA)
, Stream Analytics streams data over the cloud and uses it for near real time analysis. ASA compares new data coming in (by using the Data Factory “Move it!” to periodically move data copies from SQL DB to blobs for consumption) against historical data to simplify analysis and to identify outliers. Stream Analytics is easy use and offers many possibilities. Less than 15 minutes to set up a data connection for temperature and throw together a graph for visual consumption.

Power BI
Power BI brings visualization to your analysis through interactive reports within a browser, tablet, or mobile device. If you are a fan of PowerPivot or the eye-pleasing graphs out of Excel, you’ll enjoy the capabilities of Power BI. Show the C_Suite KPIs, top customers, and gross margins by product – all on one page.

Speech is not the only way that you can interact with Cortana. Cortana can analyze visually as well audibly.
How about an app that can predict your age based on a picture of your face using Cortana?
What about QA tests based on comparing current images to approved sample images?.

Machine learning and telemetry. Microsoft itself collects data on what functions you use in their software and how you use it to better optimize the software and the use interface. . It does not collect data on personal use.


Comments are closed.