011 Big Data And Cloud Computing – A Comprehensive Guide
1. Objective
1. 目標
This cloud computing tutorial for Big data and cloud computing will help you in learning Big data with Cloud technology to understand what is cloud storage, Big data in the cloud, characteristics of cloud computing, cloud computing services and cloud hosting, cloud data storage and deployment models, cloud computing companies and cloud service providers, cloud infrastructure, advantages of cloud computing and issues with cloud computing.
這個面向大數(shù)據(jù)和云計算的教程將幫助你用云技術(shù)學(xué)習(xí)大數(shù)據(jù)椎咧，了解什么是云存儲、云上大數(shù)據(jù)、云計算的特點双饥、 云計算服務(wù)和云托管悴务、云數(shù)據(jù)存儲和部署模式俭正、云計算公司和云服務(wù)提供商性昭、云基礎(chǔ)設(shè)施撇眯、云計算的優(yōu)勢和云計算的問題.
Big Data and cloud computing
2. Introduction to Big data and Cloud Computing
2. 大數(shù)據(jù)涤躲、云計算介紹
Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet). It’s a virtualization framework.
It is like resource on demand whether it be storage, computing etc. Cloud follows pay per usage model. You need to pay the amount of resource you use.
This computing service by cloud charges you based only on the amount of computing resources we use. So for example, if you want to give demo to a client on a cluster of more than 100 machines and you do not have so many machines currently available with you, then in such case cloud computing plays a very important role.
Cloud plays an important role within the Big Data world, by providing horizontally expandable and optimized infrastructure that supports practical implementation of Big Data.
云計算是指通過網(wǎng)絡(luò) (通常是互聯(lián)網(wǎng)) 作為服務(wù)提供的計算資源 (硬件和軟件) 的使用.這是一個虛擬化框架.
無論是存儲棺耍、計算等，它都像按需資源一樣种樱，云遵循按使用付費的模式.你需要支付你使用的資源數(shù)量.
云計算服務(wù)僅根據(jù)我們使用的計算資源數(shù)量向您收費. 例如蒙袍，如果你想在 100 多臺機器的集群上給客戶端演示，而你目前沒有這么多機器可用, 在這種情況下嫩挤，云計算就扮演著非常重要的角色.
云在大數(shù)據(jù)領(lǐng)域發(fā)揮著重要作用害幅，它提供了橫向擴展和優(yōu)化的基礎(chǔ)設(shè)施，支持大數(shù)據(jù)的實際實施.
3. Cloud Computing and Big Data
3. 云計算岂昭、大數(shù)據(jù)
In cloud computing, all data is gathered in data centers and then distributed to the end-users. Further, automatic backups and recovery of data is also ensured for business continuity, all such resources are available in the cloud. We do not know exact physical location of these resources provided to us. You just need dummy terminals like desktops, laptops, phones etc. and a net connection.
There are multiple ways to access the cloud:
- Applications or software as a service (SAAS) ex. Salesforce.com, dropbox, google drive etc.
- Platform as a service (PAAS)
- Infrastructure as a service (IAAS)
在云計算中以现，所有的數(shù)據(jù)都集中在數(shù)據(jù)中心，然后分發(fā)給終端用戶.此外约啊，為了業(yè)務(wù)連續(xù)性邑遏，還確保了數(shù)據(jù)的自動備份和恢復(fù)，所有這些資源都可以在云中獲得.我們不知道提供給我們的這些資源的確切物理位置.你只需要像臺式機恰矩、筆記本電腦记盒、手機等虛擬終端和網(wǎng)絡(luò)連接.
訪問云有多種方式:
- 應(yīng)用程序或軟件即服務(wù) (SAAS) Salesforce.com 、 dropbox 外傅、 google drive 等.
- 平臺即服務(wù) (PAAS)
- 基礎(chǔ)設(shè)施即服務(wù) (IAAS)
4. Features of Cloud Computing
4. 云計算的特點
Let us see few features of cloud computing:
讓我們看看云計算的幾個特點:
a. Scalability
a. 可擴展性
Scalability is provided by using distributed computing
分布式計算提供了可擴展性
b. Elasticity
b. 彈性
Customers are allowed to use and pay for only that much resource which it is using. In cloud computing, elasticity is defined as the degree to which a system is able to adapt to workload changes in an autonomic manner, so that at any time the available resources match the current demand as closely as possible.
客戶只允許使用和支付它正在使用的那么多資源.在云計算中纪吮，彈性被定義為系統(tǒng)能夠以自主的方式適應(yīng)工作負載變化的程度, 因此，在任何時候萎胰，可用資源都盡可能地與當(dāng)前需求相匹配.
c. Resource Pooling
c. 資源池
Same resources are allowed to be used by multiple organizations. The computing resources are pooled for serving various consumers via multi-tenant model, with different resources dynamically assigned and reassigned according to consumer demand.
多個組織允許使用相同的資源.通過多租戶模型將計算資源匯集起來碾盟，為不同的消費者提供服務(wù)，并根據(jù)消費者的需求動態(tài)分配和重新分配不同的資源.
d. Self service
d. 自助服務(wù)
Customers are provided easy to use interface through which they can choose services they want. A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed without requiring human interaction.
為客戶提供了易于使用的界面奥洼，通過該界面他們可以選擇他們想要的服務(wù).消費者可以根據(jù)需要單方面提供計算能力巷疼，如服務(wù)器時間和網(wǎng)絡(luò)存儲，而無需人工交互.
e. Low Costs
e. 低成本
It charges you based only on the amount of computing resources we use and you need not buy expensive infrastructure. Pricing on a utility computing basis is usage-based and fewer IT skills are required for implementation.
它只根據(jù)我們使用的計算資源數(shù)量向您收費，您不需要購買昂貴的基礎(chǔ)設(shè)施.基于效用計算的定價是基于使用的嚼沿，實施所需的 IT 技能更少.
f. Fault Tolerance
f. 容錯性
Allows recovery in case of a part in cloud system fails to respond.
允許在云系統(tǒng)中的某個部件無法響應(yīng)的情況下進行恢復(fù).
5. Cloud Deployment Models
5. 云部署模式
There are mainly 2 types of cloud deployments models:
云部署模式主要有兩種:
Public cloud – A cloud is called a “public cloud” when the services are open over a network for public use.
Private Cloud – Private cloud is operated solely for a single organization, whether managed internally or by a third-party, and hosted either internally or externally.
公共云 -- 當(dāng)服務(wù)通過網(wǎng)絡(luò)開放供公共使用時估盘，云被稱為 “公共云”.
私有云 -- 私有云僅針對單個組織運行，無論是內(nèi)部管理還是由第三方管理骡尽，并在內(nèi)部或外部托管.
6. Cloud Delivery Models
6. 云交付模式
Cloud services are categorized as below:
- Infrastructure as a service (IAAS): It means complete infrastructure will be provided to you. Maintenance related tasks will be done by cloud provider and you can use it as per your requirement. It can be used as public and private both.
Examples of IaaS are virtual machines, load balancers, and network attached storage. - Platform as a service (PAAS): Here we have object storage, queuing, databases, runtime etc. All these we can get directly from the cloud provider. It’s our responsibility to configure and use that. Providers will give us the resources but connectivity to our database and other similar activities are our responsibility. Examples of PaaS are Windows Azure and Google App Engine (GAE).
- Applications or software as a service (SAAS) ex. Salesforce.com, dropbox, google drive etc. Here we do not have any responsibility. We are using the application that is running on the cloud. All infrastructure setup is the responsibility of the service provider. For SaaS to work, the infrastructure (IaaS) and the platform (PaaS) must be in place.
云服務(wù)分類如下:
- 基礎(chǔ)設(shè)施即服務(wù) (IAAS): 這意味著將向您提供完整的基礎(chǔ)設(shè)施.云提供商將完成與維護相關(guān)的任務(wù)遣妥，您可以根據(jù)自己的要求使用它.公共和私人都可以使用它.
IaaS 的例子包括虛擬機、負載均衡器和網(wǎng)絡(luò)連接存儲. - 平臺即服務(wù) (PAAS): 這里我們有對象存儲攀细、隊列箫踩、數(shù)據(jù)庫、運行時等谭贪，所有這些都可以直接從云提供商那里獲得.配置和使用它是我們的責(zé)任.提供商將為我們提供資源境钟，但我們有責(zé)任連接我們的數(shù)據(jù)庫和其他類似活動.Windows 的例子有 Windows Azure 和 Google App Engine (GAE).
- 應(yīng)用程序或軟件即服務(wù) (SAAS)在這里，我們沒有任何責(zé)任.我們正在使用在云上運行的應(yīng)用程序.服務(wù)提供商負責(zé)所有基礎(chǔ)設(shè)施的設(shè)置.SaaS 要工作俭识，必須有基礎(chǔ)設(shè)施 (IaaS) 和平臺 (PaaS).
7. Cloud for Big Data
7. 云之大數(shù)據(jù)
Below are some examples of how cloud applications are used for Big Data:
**IAAS in a public cloud: **Using a cloud provider’s infrastructure for Big Data services, gives access to almost limitless storage and compute power. IaaS can be utilised by enterprise customers to create cost effective and easily scalable IT solutions where cloud providers bear the complexities and expenses of managing the underlying hardware. If the scale of a business customer’s operations fluctuate, or they are looking to expand, they can tap into the cloud resource as and when they need it rather than purchase, install and integrate hardware themselves.
**PAAS in a private cloud: **PaaS vendors are beginning to incorporate Big Data technologies such as Hadoop and MapReduce into their PaaS offerings, which eliminate the dealing with the complexities of managing individual software and hardware elements. For example, web developers can use individual PaaS environments at every stage of development, testing and ultimately hosting their websites. However, businesses that are developing their own internal software can also utilise Platform as a Service, particularly to create distinct ring-fenced development and testing environments.
**SAAS in a hybrid cloud: **Many organizations feel the need to analyse the customer’s voice, especially on social media. SaaS vendors provide the platform for the analysis as well as the social media data. Office software is the best example of businesses utilising SaaS. Tasks related to accounting, sales, invoicing and planning can all be performed through SAAS. Businesses may wish to use one piece of software that performs all of these tasks or several that each perform different tasks. The software can be subscribed through internet and then accessed online via any computer in the office using a username and password. If needed, they can switch to software that fulfills their requirements in better manner. Everyone who needs access to a particular piece of software can be set up as a user, whether it is one or two people or every employee in a corporation that employs hundreds.
以下是云應(yīng)用程序如何用于大數(shù)據(jù)的一些示例:
公有云中的 IAAS: 將云提供商的基礎(chǔ)架構(gòu)用于大數(shù)據(jù)服務(wù)慨削，可以獲得幾乎無限的存儲和計算能力.企業(yè)客戶可以利用 IaaS 創(chuàng)建經(jīng)濟高效、易于擴展的 IT 解決方案套媚，其中云提供商承擔(dān)管理底層硬件的復(fù)雜性和費用.如果業(yè)務(wù)客戶的運營規(guī)模波動缚态，或者他們希望擴大規(guī)模，他們可以在需要時利用云資源堤瘤，而不是購買云資源, 硬件本身的安裝和集成.
私有云中的 PAAS: PaaS 供應(yīng)商開始將 Hadoop 和 MapReduce 等大數(shù)據(jù)技術(shù)整合到他們的 PaaS 產(chǎn)品中玫芦，這消除了管理單個軟件和硬件元素的復(fù)雜性.例如，web 開發(fā)人員可以在開發(fā)本辐、測試和最終托管網(wǎng)站的每個階段使用單獨的 PaaS 環(huán)境.然而桥帆，正在開發(fā)自己內(nèi)部軟件的企業(yè)也可以利用平臺即服務(wù)，特別是創(chuàng)建不同的環(huán)網(wǎng)化開發(fā)和測試環(huán)境.
混合云中的 SAAS: 許多組織認為有必要分析客戶的聲音师郑，尤其是在社交媒體上. SaaS 供應(yīng)商為分析和社交媒體數(shù)據(jù)提供了平臺.辦公軟件是使用 SaaS 的企業(yè)的最佳例子.與會計环葵、銷售调窍、開票和計劃相關(guān)的任務(wù)都可以通過 SAAS 執(zhí)行.企業(yè)可能希望使用一個執(zhí)行所有這些任務(wù)的軟件宝冕，或者使用幾個執(zhí)行不同任務(wù)的軟件.該軟件可以通過互聯(lián)網(wǎng)訂閱，然后使用用戶名和密碼通過辦公室的任何計算機在線訪問.如果需要邓萨，他們可以以更好的方式切換到滿足需求的軟件.每個需要訪問特定軟件的人都可以作為用戶來設(shè)置地梨，無論是一兩個人，還是一家擁有數(shù)百名員工的公司.
8. Providers in the Big Data Cloud Market
8. 大數(shù)據(jù)云計算市場供應(yīng)商
Cloud computing companies come in all shapes and sizes. All large software vendors either have already started offerings in cloud space, or are in the process of launching one. In addition there are many startups that have interesting products in cloud space. Here we have a list of major vendors of cloud computing. Few of the cloud providers are google, citrix, netmagic, redhat, rackspace etc. Amazon (aws) is the leading cloud provider amongst all. Microsoft is also providing cloud services and it is called as azure.
Infrastructure as a Service cloud computing companies:
各種規(guī)模的云計算公司都有.所有大型軟件供應(yīng)商要么已經(jīng)開始在云空間提供產(chǎn)品缔恳，要么正在推出產(chǎn)品.此外宝剖，還有很多初創(chuàng)公司在云領(lǐng)域推出了有趣的產(chǎn)品.在這里，我們列出了云計算的主要供應(yīng)商.很少有云提供商是 google歉甚、 citrix 万细、 netmagic 、 redhat 纸泄、 rackspace 等. Amazon(aws) 是所有云提供商中領(lǐng)先的.微軟也提供云服務(wù)赖钞，被稱為 azure.
云計算公司的基礎(chǔ)設(shè)施即服務(wù):
Amazon’s offerings include S3 (Data storage/file system), SimpleDB (non-relational database) and EC2 (computing servers).
Rackspace’s offerings include Cloud Drive (Data storage/file system), Cloud Sites (web site hosting on cloud) and Cloud Servers(computing servers).
IBM’s offerings include Smart Business Storage Cloud and Computing on Demand (CoD).
AT&T’s provides Synaptic Storage and Synaptic Compute as a service.
Amazon 的產(chǎn)品包括 S3 (數(shù)據(jù)存儲/文件系統(tǒng)) 腰素、 SimpleDB (非關(guān)系數(shù)據(jù)庫) 和 EC2 (計算服務(wù)器).
Rackspace 的產(chǎn)品包括云驅(qū)動器 (數(shù)據(jù)存儲/文件系統(tǒng)) 、云站點 (云上托管的網(wǎng)站) 和云服務(wù)器 (計算服務(wù)器).
IBM 的產(chǎn)品包括智能業(yè)務(wù)存儲云和按需計算 (CoD).
AT&T 提供突觸存儲和突觸計算即服務(wù).
Platform as a Service cloud computing companies
云計算公司平臺即服務(wù)
Googles AppEngine is a development platform that is built upon Python and Java.
com’s provides a development platform that is based upon Apex.
Microsoft Azure provides a development platform based upon .Net.
Google AppEngine 是一個基于 Python 和 Java 的開發(fā)平臺.
Com 提供了一個基于 Apex 的開發(fā)平臺.
微軟 Azure 提供了一個基于.Net.
Software as a Service companies
軟件即服務(wù)公司
In SaaS, Google provides space that includes Google Docs, Gmail, Google Calendar and Picasa.
IBM provides LotusLive iNotes, a web-based email service for messaging and calendaring capabilities to business users.
Zoho provides online products similar to Microsoft office suite.
在 SaaS 中雪营，谷歌提供了包括谷歌文檔弓千、 Gmail 、谷歌日歷和 Picasa 在內(nèi)的空間.
IBM 為業(yè)務(wù)用戶提供了基于 web 的消息傳遞和日歷功能 LotusLive iNotes.
Zoho 提供類似 Microsoft office 套件的在線產(chǎn)品.
9. Issues in Using Cloud Services
9. 使用云服務(wù)時的問題
Some important cloud services issues are as listed:
列出了一些重要的云服務(wù)問題:
a. Data Security
a. 數(shù)據(jù)安全
Organizations must ensure that their agreement with the cloud service provider ensure data security. Handing over private data to others worries some people. Corporate executives might hesitate to take advantage of a cloud computing system because they can’t keep their company’s information under lock and key.
公司必須確保與云服務(wù)提供商的協(xié)議確保數(shù)據(jù)安全.一些人擔(dān)心將私人數(shù)據(jù)交給其他人.企業(yè)高管可能會猶豫是否利用云計算系統(tǒng)献起，因為他們無法將公司的信息保密.
b. Performance
b. 性能
Parameters of cloud performance must be specified in the agreement and quantified wherever possible. Exceptions must be clearly noted. Service-Level Agreement (SLA) should clearly state all the terms and conditions between a service user and a service provider to ensure proper performance.
必須在協(xié)議中指定云性能的參數(shù)洋访，并盡可能量化. 必須明確指出例外情況.服務(wù)級別協(xié)議 (SLA) 應(yīng)明確說明服務(wù)用戶和服務(wù)提供商之間的所有條款和條件，以確保適當(dāng)?shù)男阅?
c. Compliance
c. 合規(guī)性
Cloud services must be compatible with the compliance needs of the business. Some companies are also concerned about regulatory issues. Market observers say that around 50 percent people worry that they will be tied to one provider of cloud storage.
云服務(wù)必須與業(yè)務(wù)的合規(guī)性需求相兼容.一些公司也擔(dān)心監(jiān)管問題.市場觀察人士說谴餐，大約 50% 人擔(dān)心他們將與一家云存儲提供商聯(lián)系在一起.
d. Legal Issues
d. 法律問題
Organization must ensure that the location of the physical resources of the cloud does not bring any legal issue. The cloud presents a number of legal challenges towards privacy issues involved in data stored in multiple locations in the cloud, additionally increasing the risk of confidentiality and privacy breaches.
組織必須確保云物理資源的位置不會帶來任何法律問題.云對存儲在云中多個位置的數(shù)據(jù)涉及的隱私問題提出了一些法律挑戰(zhàn)姻政，此外還增加了保密和隱私泄露的風(fēng)險.
e. Costs
e. 成本
Organizations should be aware of all the costs involved with the use of cloud, and use the services in a controlled manner as cloud offers pay as per usage method of the cost incurred by the company.
組織應(yīng)該了解使用云所涉及的所有成本，并以受控的方式使用服務(wù)岂嗓，因為云按照公司發(fā)生的成本的使用方法提供支付.
https://data-flair.training/blogs/big-data-and-cloud-computing-comprehensive-guide
