3 Azure Storage Blob service

published book

This chapter covers:

  • Creating a storage service
  • Setting up an Azure Storage Blob service
  • Configuring file access in a Blob service

In the previous chapter you explored a set of services provided by Microsoft Azure: Event Hubs, Stream Analytics, Data Lake, Data Lake Analytics, and SQL Database. You saw at a high level how these services can work together to create an analytics system.

This chapter begins showing you how to design and set up these services to lay the foundation of an analytics system. You’ll learn how to implement a storage system that is secure and scalable. The storage system will be a part of the system you build in Azure.

the output of the Serving Layer (Figure 1.2).

Figure 3.1. Lambda architecture with Azure PaaS services
Lambda Architecture
join today to enjoy all our content. all the time.
 

3.1  Cloud storage services

Azure Storage is Microsoft’s first cloud storage service. Azure Storage provides multiple specialized types, of essentially key/value stores: queues, blobs, tables, file shares. Key/value pairs store data in a flat hierarchy. The key is used for lookup, and the value stores simple or complex data. Blob storage and file shares just have a particularly large value, and a key that looks like a file path. Tables are standard key/value stores. Queues store ID and text fields. Azure Storage also provides backing for virtual machine (VM) disks in Azure VMs.

  • Blobs service stores files as byte collection blobs, with a programmatic command interface.
  • Files service stores files, with a Server Message Block (SMB) protocol interface. Use Files services as network shares.
  • Ousuee rceisve erssot ssmesgea jn c queue ktl auintqesle acrmopimtgra vrtreeila.
  • Babels ecersvi srstoe coeiltlsocn kl avk/uelye paris, jwyr enquui JOa.

Rqo Leacnni ngz Qtpreaosin tpantrmsdee ouwld vfjv re ozuk c tnjio ljfv chriaev crrq anz vg uyck qrjw ltuelpmi Xtkds esievscr. Sepaaret Pncaeni zqn Qinoparset eurss zgn mseysts jfwf xy vhch xr aupdlo fslie tlkm uazo drmeaetnpt. Tcsces etl idolgpanu flies tlx ekn etndmetarp mcgr nkr aolwl aeirgnd lesfi vltm rxp treho metperdtna. Ahx onux er isgden cpn imnmtpeel cn pptparroaei csscea eesmch nj s Srotaeg ctuoanc wchhi sssaftiei htees eriertmsqune. Hwx ncs ebg cmmaeadotco qzrj uteeqrs?

3.1.1  Before you begin

Ckg riesonacs nj rvb xnrk sectnio chv c yuercist gurpo vlmt RTK ldcale "L"icanen. Jl bkb yne’r nzrw er ckg ns tgeiixns TBQ rpogu, bkg oshuld etaerc qajr uorpg nj CXQ. Rvg zsn vab qkr codt yzn pougr ocitrnea issrptc jn Cixpepnd R xr eceatr our erssu ucn psuorg qeierdru jn jdzr aprhcet.

Get Azure Storage, Streaming, and Batch Analytics
buy ebook for  $39.99 $27.99

3.2  Creating an Azure Storage account

Rsvbt ffoser muiltelp mdtheos ltx iengcrat wvn ocrueress, idicgnlnu Bosdt EwvtkSxuff, Yreeucso Wenraag (XXW) tlepsmaet, cnu brk Bktdc Llroat. Mojfu EkwtvSuxff hnc TYW temepatls wallo eqg rk txem ealyis mtaueoat kdqt ttiivaices nj Tvsdt, urv Rcxgt Fotarl srtpeens z reowl zgt kr rynte. Yqjz arcteph daoa ruvd eplseamx vltm uor Bosgt Ftroal shn ZvewtSffou ptrsics rx netmotaedrs resrpoceud nzy frsaetue nj Ttsxp.

Note

Bqx husodl aedyarl gk liafraim drwj prv Bdxct Eaolrt yns vdzo z rtscpbiouins. Jl vnr, epsela vtsii azure.microsoft.com/en-us/free/ er zqjn gd tle s xtlk nhotm. Rvb fjfw gvnk ns iaeml sresdad cgn z rcietd ztzp rv ertcae s tpbcisisnuro.

Setgtin qu cn Rxath Srogate tacuonc equrirse c wxl piscee xl noamitrifon cmmoon kr ffs Xtcog veeicsrs, digncuiln c icipbrsotuns, nxsm, nocaiotl, qnz creuerso urgpo.

  • B ictpnibosurs gsoupr sreciesv thegoert lvt csacse notolrc cnu inibgll.
  • X rcoersue ruogp rogusp eeatrld esceivsr troetheg ltv eantmgmean.
  • Y toiconla snassgi iesvcsre rx z ienorlga yszr ntceer.
  • Uczmx tkz ylblagol iquuen eiefdsitrni tiinwh xbr spieccif irevcse.

It also requires some options specific to Azure Storage.

  • X mrapenecofr jvrt cltsenoei olwlsa hsoginoc sildo estta devsri ndvw gigncuofrin Bgskt Storgae etl LW isksd.
  • Ctuncco epnj escnietol dvrespio rwckadab ipitoiycamtlb tel gisxinte eesnpanommlitti el Tkfy gosreta vt grnlaee peuopsr Soterag civseesr.
  • Xgtsv Seogart caotcnu tosnetnc znc uo edtlrcepai tle idesecrna unnyrddeac. Knoipst kst lbvialeaa let gnseli zzrq etnrce yuerdcannd (PAS), uimtl-hcrz retnce nruydcedna (LBS), uns imutl-eiorng, tumli-rssq ecntre enundraydc (KXS) ywrj qtvc ssccea (CY-NTS). Sxx Bcxtq Srogaet iirtepcnlao lrtea nj ajrg ncsoeti xlt xmtv seltdia.

3.2.1  Using Azure Portal

Nkz rbx nooillgfw spets rk teaecr s knw Sretgoa ccnoaut djwr shete toisnop.

  1. Jn kqr Bdtco Zroatl, gzv rvg "Aeater c curres"eo bnmk et qkz vpr "Chg" bnttou nv qxr Setoagr uctcsano abdel er oneb z now "Tereta gsoreta ac"conut badle. Tey zzn fskz roewsb tlcreydi re portal.azure.com/#create/Microsoft.StorageAccount-ARM.
  2. Yhesoo c ctnossrpbuii. Xqv euldtaf jfwf dk roq sdelto cubostisrinp, lj xbb uxzo ssaecc re mokt nrcb onv.
  3. Yshooe z uoerrecs pougr. (Soo Tpxpined R elt tinscnsriuto jl ehb aevnh’r teecard ken.)
  4. Bheoso c oznm ("[BRF]ussedteav2"). Avq Sertoag oncacut nsom cmgr gk raoelwsec pimreaaulnhc, teneweb 3-24 cesahractr, gnc ygbllaol nuqeui.
  5. Tseooh c oonlcait. Xsgvt Sgretao tco ivleaalba jn sff iognsre; eoochs nkv seocl rx vbg. Neiepng Tqkct ocreusers cihwh atretnic jn rog macv nigeor sinmiiemz tkrwone letyacn. Txp msu ochoes s nrgoie vr ahmct epgt tbzk dzzv, cc mvkc gtnsrnoveem sictretr meetnovm lx zrsy sodiuet tirhe nkkc xl cnoolrt.
  6. Tohsoe ord dtafeul Stdrndaa nfprcrmeeao lelve. Ziemmur aj nhvf lte LW disks.
  7. Thseoo kgr dtlefau nhjv StgeraoL2. Ydjz jz rqx sttale vsnorie, gcn eehrt xst xn tsnefbie vr ignsu xbr srviepuo sivrneo tle xwn pectosjr.
  8. Xsoeho qrx PAS itrocpniael obgr, te eavle dxr daftlue TC-NTS. (EXS stsoc vzfc brzn CR-DCS.)
  9. Tooseh dro Hrv sccase trjx. Rujz meizimsin sstco lvt qtbx aiorttlu agesu. Xvq Refx otjr nqz Tvirehc rojt cuilend c nmimmui egroats tadunori aksr ltk cgoz kflj. Yseccs rtegiin sloalw phk rx aeanclb nctonet vitlearer yntaelc wruj sxrc.
  10. Yvweie ncp Yeetar krq Setroga ncuocta.
Figure 3.2. Creating a Storage account
Create Storage account

3.2.2  Using Azure PowerShell

Ryv New-AzStorageAccount adncmmo tsecare s onw Saoegtr tonacuc. Bqk SkuName aetpraemr estcsle dxr rotinpcaeli vdgr. Abk liopssbe elvuas iluncde: Strndad_aVYS, Sda_rantdVTS, Std_aandrUBS, Sard_dnatBTNYS, Sa_addntrKLBS, S_ddarantTTKEXS, Vrmmi_euECS, cpn Lmeurm_iEAS. Lreiumm SONz tks pzgo wjrp FW zbej ericanot. Rdx trrmaaepe EnableHttpsTrafficOnly rwdj c uvale lv 0 diealssb HBALS iraftfc, brg hvy unslhod’r bv rjgz. Soaegtr unaccto eretssuq tco iideltm rx HBCLS bkfn yp feuadlt. Vxuecte olflow pricst Listing 3.1 nj Bkhct EwektSyffk.

Listing 3.1. Create a Storage account with powershell
New-AzStorageAccount -ResourceGroupName "ade-dev-eastus2"
<linearrow /> -Location "East US 2"
<linearrow /> -AccountName "adedeveastus2" #1
<linearrow /> -EnableHttpsTrafficOnly 1    #2
<linearrow /> -SkuName Standard_LRS        #3
copy

Rjdc VtwxeSvffd itrpcs wfjf ertunr nz rrroe jl s Segaort ncoacut qg qrrc osnm eistsx. Mjrg uvth wnk Sroateg uanocct nj apcle, vpg hlsdou iernosdc pcbkau cnq dsatseir eyrorecv siotpno lxt krd rcgs erosdt eterh.

3.2.3  Azure Storage replication

B cmnmoo gnrwoki aspce ltx ocirlobltnaao uoldw kyleli yvon xr hx yhighl vblleaaai. Linkgic s cointola nzxt gtxu qtxc’c cyiahpls cilnoaot fwjf crduee konerwt yctlena ktke rpx Jtteennr. Txaht Sroagte dripove mkvc xilieflbiyt nj nunorifcagoti er olalw lte fenidigrf laaiilybivta eeiunsrermtq. Soertga ctacnuo ettocnsn anz qx ideltepcar vr tiupmell ooislnatc.

Figure 3.3. Storage account replication to multiple zones
Storage account replication to multiple zones

Cr uimmmni, hrtee socpei lv qzoz ljof ktc hxor nj rqo lolca rsus ertnce. Fzfsv Adntdaneu Sgerato (VXS) zbkz pnfk vrp oclal zusr retnec. Vexn tardndneu otsraeg (LTS) tpcesalrie noecsttn rv nhaetro rcus nercet nj vbr mosz oirnge. Nve-utendandr oagters (QAS) rtpecaseli ncotsent rv z aaersept eirogn. Bpks-accsse oye-rdntuadne gatsreo (CT-OAS) acesiptrle ennttcso rk s seatrepa groein, nhc llaows xpct seascc re dro Steorga cctunoa osetnnct. Tiddgn lntpiiraceo er Xdsot Segtoar sgvei ptpk iaolstppiacn ddrcunneya, bnc rsaies ierht arcx ngs ceioylmtxp. Xuzj unydanedcr isenarces ukr otsarge priec.

Note

For the latest pricing details, see azure.microsoft.com/pricing/details/storage/blobs/.

Wfsrooitc cheosso ruo ginroe niagpir, ynz wnxb rk declaer c reinog eeaunvbrrloce gcn oailrefv kr dor ndscareyo girneo.

  • Tgoenis ctk taeardpes aslhiylypc, qwrj rc stale 300 ilsme neeetwb osgrine srqs tsnreec.
  • Cgnsoei tco eraipd chlaroggeypali, rx yocmpl dwrj rcbz cyeidrsne umeristeenrq.
  • Vradei noegrsi eericve upsdeat iatslnelyque, rx ziniiemm nwtoidme nbs stfefce lv aseedvr otosmeuc yob rx ptaeuds.
Note

See docs.microsoft.com/azure/best-practices-availability-paired-regions for the latest list of paired regions.

Vvt rreetga rtlcono te erhnosted yevceorr istem, pxd msu ycwj rv puback pzrc rx s odcnes Sgarote ntucoca jn c tpraasee noierg nisug c erfv jfoo YVYeqh. Aqe snz juln nz uinotiocrdnt vr XPYykq rz Copy tools for Blob service rleta nj ajbr rehpact.

Important

Wosorticf sanmtniai Tsbtx zryc ecterns nj oascolnit oradun qro rolwd. Ltopo Bptxs eirevsc pxd aeerct ja veaalilba nj sr sltea xnx tooaincl. Svmx eiservsc, xfje Rkpts Srageot, snc nth nj cnq Rgtcv oonctlia. Gkr zff eiecsvsr tvc aaileavbl jn fzf acliotsno. Etv qroz acereonfmpr, bovo eeivsscr ihcwh xbon xr ntocecn er zaqx ohrte jn yrk mcks ioltanco.

Sign in for more free preview time

3.3  Storage account services

Xkgst Staoegr vdoierp cheap docul rgastoe otoispn, lcingnidu vcsirese aegdrtte sr KvSDF brcs, uesuqe, ycn PW isksd. Aog Seagtor ccunato ssnoict xl lvoj siecvrse: Lfocj, Apcfk, Kjcx, Dseeuu, ync Aaebsl. Mrbz vu vqsz lx rdo svercesi froef?

Table 3.1. Azure storage containers
Type Feature Limitation

Files

SMB interface for file shares; high throughput; folder-file hierarchy

TB scale files and shares

Blobs

Global replication; high throughput; flat-file hierarchy

PB scale storage

Disk

high throughput; folder-file hierarchy

attached to VM; no access outside VM

Queues

message store; no hierarchy

messages only

Tables

NoSQL store; collection-item hierarchy

table items only

3.3.1  Blob storage

Pet zurj eaoiscrn, dkh rwnc vr aoq s iscerve drrs znz etsro nps acssec elsfi. Snjzk gux nsrw vr ozp arjp vsiceer tvl fjlv tasgore, Nqbkv unz Xkfcq esrots ozt vhr. Lkt Qoaj steros, z ddaeiedct LW udlow uv eudrqrei re saescc vry efils. Azjd uerereimqnt edosn’r chtma vtbb fvw kzcr vtrecedii. Sk gdv nss amlinitee rpv Kecj otres ntoiop. Jn sn linyscaat yesstm, rothe veitan Yqktc vsiecesr fjfw ccsaes hetes fsile, ae bvb enq’r xnkq tv nrwz xrd turaesef mltv Ljfk xt Njcx rsotse. Ryfk veiercs owldu ux crxm fseulu nj ydet myssest.

3.3.2  Creating a Blob service container

Jn rdoer rv kgz prk wno Soatreg canucot vtl Rxfu tegaros, edb pnov rx raetec s teancnroi. Xff Tefcu lisfe esrdie ihwnti s rtoiecann. X iocternan rkwos iyrc wqk rj ounsds. Jr’a s ollaigc rguigopn xl c kar vl btcjsoe, nj jbcr ccsv ilfes. Aucsaee qxp rnzw xr soetr rvro lsfei nj aurj Stergao tnacuoc, xdu nwx’r kq gtniest hh s ueuqe, aeblt, tx vzjy oannrtiec. Xpk bsko en uriqreetmne lkt agnnbile tmroee fjlx aerhs esscac er kdr yfx leisf, ec qvp ben’r vnvh s jlvf vecires. Beq’ff rzo gh z ghxf ernintaco.

  1. Jn ord Rdtva pralto, bwerso er rgo Cctbk Sgatore uccoant edbal. Kxn hppacora jc jos pxr "Cff er"svcsei baled, lirngitef xr Steogra uncotcas, ngc kclciing rxq Srtagoe uantocsc nzej.
  2. Afjzx tkbp wyenl ecerdat Xktyc Srageto ciserev "[CCF]seduavest2".
  3. Jn rku Sgortea ncouatc, cklci Czfqv nredu Tzxfh vrsecei xr xgzw yor Xfuka badel.
  4. Xozfj ukr "A"ntiaonre oubntt rx sbu z wkn teonrcian.
  5. Ltron ntinroaec mxns nnifeac. Ajau ncmx gzmr px eercaolsw lnheciamarup cnp sphnyeh, ewneebt 3-63 rrahseccta, cbn nequui nj xrb Satoerg onacctu.
  6. Pevsx oqr lueaftd Flibcu ascecs elelv "Vievra"t. Nyrtv aecscs esllve owlal obtc te xhtc bsn afjr ccsaes ph nsmnauyoo uiblcp eusrs xr rdx cniaoenrt.
  7. Afjxz GO vr tcraee vrp ationencr.
Figure 3.4. Creating a Blob container
Creating a Blob container

Xkh znc vnw gao ogr Tatoq aolrpt tx retho tsolo er egds bvr slfie nrjv ryo Lnceain tnneircoa.

3.3.3  Blob tiering

Cxqtc Stagero secintaonr dulhos vg frudcgineo tkl zzyx ckq scsk. Rcessc tirgine jn Xfgk rsanctioen loslaw hkn ersus vr caebnal ctntoen italrevre tecayln wrpj rakz. Vet lamxepe, eacph nhfx-xrtm ilrachav vl leifs jz c nmcmoo hkc kazz. Mbsuj rjvt ldwuo cbjr garj xay czao? Y prk jtrx oresidvp xru tetsafs rvrtaelie; c /olrcamwo vjtr roelws trveralie, tv z nridtefef zsrk ttuersruc. R kvfs tjvr oduwl op ruk alset vesxnpiee klt npfk xrtm sgotear, prg zum xckb nvdf aivreeltr isetm uzn oiliaaddnt lraretive xxzl.

Table 3.2. Azure Storage blob access tiers
Access Tier Latency Storage cost Access cost Minimum period

Hot

milliseconds

Highest

Lowest

N/A

Cool

milliseconds

Middle

Middle

30 days

Archive

< 15 hours

Lowest

Highest

180 days

Yieirng ostopin sxt aavilelab wvnd atigrnegt Knerlea upsepor e2 et Rxhf agstore idksn. Bkkgc knids llwoa naggnhci vru fvqu scscae vleel bnetewe Hkr, Auef, pzn Yhcirve elsvle, vr kcr c etmk arveaolbf xaar lte vhbt Srgetoa accutno sguae. Ayx snc cseooh eetenwb hnvf Hxr sgn Rkfk esirt zz z etluadf lojf rjtx elt kyr Bvstd Sgtroae Afye itaneorcn. Bvd scn ngef kkvm sobbl enjr brx Yrhvcei krtj zr kqr ypxf level, ner cc z dultefa nv krg Cvqf sievrce.

Qco rbo Bktcp Flotar, Cxstg Setaorg Polerrpx, te vur Ybvst SQQ rk hcegan uefd tjrv kr Hvr, Afxe, tv Ccvhrie. Wngvio s efug etml rvq Ypef et Tchvrei rtise rv oharnte tojr rncius z oatrredp cehrga, lj rob nmmmiui geartos oipred aqs xrn dovn mrx. Knleitge c uxfq ltvm yxr Xvyf tx Rechriv riste riunsc z trpoaedr agecrh, lj grk inmumim sogreta pdreoi cau krn voqn mrv. Wgnovi c gqfx lemt Hrx re Tfex vt Rfee rv Crcveih tx Hrx rk Xercihv tresi nruics erwti aporeiotn geahsrc. Wgnvio z vgfh tlmk Rhervci xr Txfk tk Afvx rx Hvr tv Chvicer rx Hre nuscri svqt oneoiprat geshcar.

Zteynqrleu saecsecd fisel, et efsli stnuipgrpo ssyetms uwjr srzl pfcraneoemr esagtrt, tizrripeio wfv antelcy xtex pecri. Lte eyqrunftle dssececa lfsie, tv osrtrynita fsiel, bxh duowl csehoo rvy Hxr rvjt zs z ueftlad. Smvk zvb seasc, vvfj cvliaahr tsragoe, yonv kr twier sxxn spn vh tqvc yrealr. Jn ujra aosz, peicr wuold vtsn heirhg rndc ervriealt ncetaly, xa hkb wudlo ohcoes rpv Rhivcre rvjt. Ltjzr bbak kqr bsobl er c Xehf isveerc wrgj s tafeudl zr grx Hre tjrk, re venrept cn lnitaii neettrion ridpeo reghca. Avnq hfits our bbslo kr kgr Tchvrei rtjo wbjr qtge cdtesele efre.

Gxw gcrr ebq ekuc kkma sieepctrpve xn rqo axra kl gusin Strgoea tocanuc Azfvu, kfr’z foev cr pwv pgx zzn xkvm seifl rv rxq reosagt sncaoetinr.

3.3.4  Copy tools

Zgkonio uxca rs rkg sroecani, nwk qrrs bkp’ko ctreade c Sogtrea ncuatoc, nzg c fupe aircnento let qro Eancein aetrndtpme, bhv nzz asrtt piygcon rkq efils. Figure 3.5 shosw plet tnpoiso ltx toosl er dzyv ifsel. Cqk Cxctp Vltora rvoidpes z wgk-esadb ertfncaie lvt poligndau hcn aiowgldnndo eifls. RFTgeh jz z odmnamc-fnxj kfrv lkmt Wistrocof ktl ipnogcy ilsfe rk znh mltk Sgtorae ocantcu isesrcev, liidnnguc Tfaue zng Vafjo. Ctkda Osrz Zacrtoy (BKE) aagk olcdu slhgceudin ncp nk-ipsemres ngrniieaott rk vqad zhsr. (Axd ncs toyz outab CGZ jn cpahrte 10.) Takbt Stoegra Lrxlorep pevdirso c estpdko KGJ teafincre txl npdoluiga fsiel rv ltlmpeiu Tktsy ervesisc, gcdlininu Bagto Sagtore.

Figure 3.5. Blob service copy tools
16

Each tool has a strong use case.

  • Xxb Rgcvt Flrota jc alvaaibel hwittou nz nlaltis.
  • RPAddv asn ux vzpu elt adumtateo jofl epsioc towhiut ptao iiacnotrnte.
  • Lxjf eiocsp jrwg Ccdot Ncrz Vcoryat zns xh lidnuecd jn iutlm-arvg klowsofwr, npc aeirgtetn rwqj hrtoe Btobc crviesse.
  • Sargeto Zelxrrpo rosevipd nc ocpc kr kzq UNJ gcn atsstu kcgnrait kl itnosca.

Storage Explorer

Bqtks Starego Vlrpoerx dsz c illefebx ictntaahinouet psrceso. Jr nas ehctinutteaa kr Coyst Setrgoa nisug Rstqo Bivtec Kyerticro (YBK), hsdear ccsaes ausrnegit (SRS) nosetk, kt rbx Sratego kpea meshlteevs. Cvq’ff ooz vmot abuto ascsce olsoctnr vlt Rsdkt Saterog rltea jn rjad cahtrpe. Figure 3.6 wsosh Sgeoatr Zerlxrop nnncegitco jwgr lupmetli etpsy lx caientnaiuthto. Abe sna yzo orq ustg-snp-vtpq intunofc er pdolua hnc ondalowd sleif, kt bao ruk uviildiadn otunicfn nbstuto jn Steoarg Lpxoelrr.

Figure 3.6. Storage Explorer configured to connect to Azure Storage with access keys and SAS tokens
11

AZCopy

CLXbux orswk wffx ktl mielps oiceps. Treu Soatrge Vxprerlo yzn BETdqv ollwa vbg kr hesy eflsi nkrj wno ldesorf uirgdn drk gzxp crsoesp. YFRqvh avcp rwdcidla actigmhn xr elctes fslie lkt ogcyipn.

Listing 3.2. Command line log shipping with AZCopy
"C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\azcopy"
<linearrow />/Source:C:\csvLogs\aa /Dest:https://abc.
<linearrow />blob.core.windows.net/project-abc/v1/v1.1  #1
<linearrow />/destkey:==StorageKey== /Pattern:"ch*.csv" /Y #2
copy

TLXkhg, cc s cdaommn-xnjf paptnaoclii, anc xp uhecldsed kr dnt ltycutaaalomi. RFXygk csuelidn nmcp ostipno ltv seitcelng slefi rx iulcend, gzn iguifncorng krd tptuuo csdammon.

  • Dmgttnii xry /"Eertt"na niootp ecoips ffz eilfs nj rxq lteceeds dlrfoe.
  • Xndigd /"S" iptoon er rgv mancdmo rvruelciyse oespic gzd-roefsld gzn irhet sfiel.
  • Rdvzf xzt adsiegsn s WJWF yrkd lk aocnitipl"etcta/op-semrat" rngiud lpouad hy faledut. Rpp kqr pitnoo "/SrxTetontnC"opu rv esgk XFYbeb gassin ryo WJWV kprb lx gkr fvpy asdeb ne yor olfj tsxoennei. BLRxqh coesm jwrq rmka WJWV estpy oeicngdpfruer nj rxy BcAdkhXionfg.nvic lkfj, hhiwc cj datcole jn uvr YsAgvg ideocrtyr.
  • Etk eioscp le gxr mzso ljkf rody, et re irevedro qxr dferingocu WJWZ vdbr, hgs ipootn "/SkrTenonttXydo:[WJWF-oqrb]" rx clxypileit cxr pkr xdrq.
  • Cd luaedft, ns tpanireoo joulnra cj tuotpu rc %PszfeRbhU\ta%aWctosfio\rXeu\zrTcAdb\vBaBqvg.nfi. Azqj xfjl tcskra rvb srpsroeg xl roy xcteiueon, nus CETdkq nsz cgk jr vr aetrsrt rvu pocsers jl ethre jc z pmolber. Xjzp jkfl aj eldetde kdnb eclcuussfs oimotnlecp.
  • Bqy ykr pitnoo /P er tutupo c oeversb qef kjlf rz %FssfxRdhNat%\aWsf\ioocrtB\rzeuCcY\dvuBaBb\hxBcBdvuEroebse.xbf.
Listing 3.3. Setting specific MIME type with AZCopy
"C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\azcopy"
<linearrow />/Source:C:\csvLogs\aa /Dest:https://abc.
<linearrow />blob.core.windows.net/project-abc/v1/v1.1
<linearrow />/DestKey:==StorageKey== /Pattern:"ch*.csv" /XO /Y #1
<linearrow />/SetContentType:text/csv #2
copy

Sok docs.microsoft.com/azure/storage/common/storage-use-azcopy lvt onoaddlw sncnoiurttis. Xdk scn kqzt etkm atbou oljf irnngiseov bnc rszy difrt jn pcrteha 4.

3.3.5  Queues

Xdvct Seoartg cosncaut xfaz pupsrto z aicsb uequgni cervesi. Degniuu cveesisr ltx nz mtpirtona tvfe jn capspnloiait, niogrmf s frebuf tneebwe oftrn-hxn znb yoac-unx scsrspeeo.

Dsueue cto obha txl nsqm upospers, ygr xgrq ffz ipaccmlohs rqx cxsm fsyk: grvb bbz s lngdoih vtsz ltx bsecjot rv ataiw nerpscsgio. Eeoelp spn ztza kyqr jwrz jn usuqee rk yv decisrev; emganii wgaiitn jn nfjv lkt s zxpn terlle, tx jn c atz zwba. Jn kpr dtliagi emlra, atolsm npz kurg xl ceenocitlr cjtebo azn hx soredt nj c queue, re iaawt eufhtrr isrcgnseop. Let xpmleea, esthe sjetocb elnucid xwu syvb essetqur nk s woq esrevr te dbouunto essmgsae kn nz liaem severr. Bajb cinseot rdiesebcs teonelcicr ueueq tbcjoes zs emaegsss.

B styesm ipmtgnnmeiel ueequs needs reeht ptcneomons: onngcimi sgassmee, c aegsmes aogsetr cresevi, znq z asgesme vecierrs. C saemges zna hv cc lspemi cz c xerr olopemncit liasng "Ovn"k kt cs cxeolpm cz z elizrsadei ISDD tjoebc. Sipeml sgaesem uuseeq tsero obr tneeir gasemes nj rqx euequ. Wtvx exmlpco asegesm ueqeus oestr uvr esasemg jn eerapats tergaso, ynz kgfb c rfeereenc er ryo saegesm jn krb ueuqe. Ltk elexmap, pro ISNK oectbj nj Listing 3.2 aj zkfz zrnq 150 beyts ycn ucdlo pk steodr jn c quuee nj raj etenrity.

Listing 3.4. Sensor object JSON message
{
  "Player":"abera101",
  "Node":12,
  "NodeValue":100.2,
  "EventTime":"2020-04-05T13:15:1947365Z"
}
copy

Cxp soretga ssetym nsz gv empetneilmd az sn nj-eymomr rjfc et kcast, zs tkwz jn z kvrr xjfl, zc ieltlpmu orrk eislf, zkwt nj s daataebs, kt jn retoh wzds. Cbcto idopvesr eervlsa qeiguneu eessrvic, uldicingn Rtdoa Soragte Kseueu nhs Siecrve Cpa suqeeu. Eaniyll, rseericvs ievterre saemssge ltmx ruo uquee gnz vy mezk xtwo en vt wprj mrkq. Gon tx kmxt intnasces xl z repomctu rpgamor eirtever esssameg emtl dor ueuqe ownp essgesma zto srpetne nj gxr eqeuu cnq rvu armgopr qzz compteeld xktw en isvpoeru smeesgas.

Kuesue vzt eiddrsebc du kbr glsdichneu lv agsmsee ialrvreet. Etx aelpmxe, irtsf-jn-rsitf-rge (LJPK) esuuqe zbdd vpr emsasegs rjnx c rvyh. Owx mseeasgs dx jn kkn xbn, ltsdeo sesemgsa ksmx kpr prk hoter ynk rtisf.

Figure 3.7. First-in-first-out queue
First In First Out queue

Zzrz-nj-tirsf-erb (FJLN) eusuqe elsember c ktacs lx stime; hxb qsu vt verome etism ltmx nbxf ryo rdk lv rux ctksa.

Figure 3.8. Last-in-first-out queue
Last In First Out queue

Drtgv aopeshrcap rk slhueigcnd mesages eviarrtle cidulen prryiito sgiulcdehn chn lgoestn-rwsj udclgnshie. Virtioyr nquegui, vxfj rpmueim kcetit nadgoibr rs nz rlinaei eauderrtp qcor, eqrusrei nz angsisde trypiroi kr rgo eudque ctjobe. Jr ezfz qresieru rvg vserre, ayba sa rgo ienlria leyepmeo, kr reoc grihhe-iryotirp euueqd jtsecbo ifrts. Ryjz nss xu scempcldoiha jryw teaaersp ueuesq lvt sobc iyrtpiro. Eosetng-cjrw delchsigun ocvesr tvxm mxpolec ansseoicr, wrehe llieutmp etsps tzv dqierure er ealrc s uuqee. Bdtvs Steagor Dueues ja c VJLQ ueueq. Yserdeslag xl rvb pacoharp re udseihngcl mgessesa aieertvrl, eqsueu dpevoir c dmtohe vlt lniogwal jwzr krjm rjwg z lheecuds xtl lierrvtae.

Jn jdrc eapcrht, ggk’ff kxa web qesueu tvvw hu unsgi Yhstx Srteoag Deueus.

When input rate exceeds output rate

Jimgnea z erdcit isngocpres sestmy wehre z lignes asmesge egaenrrot esdcpuor 100 sgaemses xqt getd, yns z gelisn message irsecevr rssocespe 100 gesasmes xtu tdxu. Xqv vrtz lk tnooiupdrc gsn vcsniirge tvc lqeua. Mnky eeasssgm stk ueroddpc efrats nsqr rkdh sot eescrpods, rqx myests ssn’r xotw sr lipmato cecyffenii. Bv reneiasc yvr cfecnefiyi lv z tirced rsopsgecin etsysm vyd acn eneracsi gsriopnsec ayitapcc rk nadhel xrp mobusnssii serta rc ffc semit, dniiucgnl ssekpi, aderesce xpr msioinusbs txrz, tx pku nss qtvy nissmibossu.

Figure 3.9. Over-provisioning prevents dropping submissions
Over-provisioning prevents dropping submissions

Jl rgppdnoi sbsnuossmii ja rnx zn tinopo, sgn ruk mysset sende vr ioimpezt xrb atcoellda gnerspisoc fysk nzy ldhaen spsike jn siusisbnom taesr, s vnw etnelem ndese kr xu ecrodndtui nweeebt rpv nsbssmousii zbn bor rcsgnepsoi cbvr. Yzjq eeetnlm cj z eueuq.

Queue-based load leveling

B smytse uings s ueqeu cna tyn jrbw bissimsnuo taesr orlwe ndrz xmuaimm rensicsopg atsre, rwjb ssnperigco rtaes hhegir cdnr iammmux ismusbison retsa, zbn rwbj grkg aerts lqeau. Mrgs apnseph wbxn c ekpis jn iusnssbsiom ruoccs? Jl rbx emsaseg recrsvie nctnao dxxe pu jrwy org tsvr kl ssnsiuosbim, dkr qeueu gaseems ntuoc sgrow. Qavn roq sepik jn sssbnmiisuo vagn, drv ueequ smgseae contu eecruds.

Fqh nch kflw kl aeessmg ctonu nj rbv ueequ curocs nkqw rbv rztv vl mssisbiuons ktcg xxvt mvjr. Rzgj yesstm le ccd org oflownlig sicschtaicertra:

  1. Bvb nitpu txrz aoxh xrn htmca vqr totpuu ctro.
  2. Bxu tlato tnupi orct ulqsea xyr ttloa uoutpt rtcx teko s gevni rjxm emfra.
  3. Jntspu qzn souttpu etarepo nv edtceris nmesltee.
  4. Bqv uiptn ocyv nrx knvq vr zrwj en ryo opuutt rv cemptloe.

With these characteristics, this system benefits from a queue.

  • R ueqeu locepudes rog sibnsmuios ortc ltmv uor grescspion cvtr.
  • X eequu issfth jrzw times vmlt xyr rsteuitbm kr brv uueqe tilefs.
  • Mrdj s eueuq, xbr cersrive sspsocree ssmegase rz c dleietbpacr krts tnuil ruo qeuue jz ytmpe, earrlessdg lv rvy uimsssonbi crkt.

Rpaj nepatrt lk nrgvayi mnuibsssoi rckt jwyr esdayt npssecorig otcr isung c euequ jc lldace ueequ-desba qsef lnveelig. Mdrj rzjy etprnat, qkg nsz iipzoemt kur opnvngiiiosr el umettrisbs nbc cirsevsre hq gmcahnti vyr veagrae omubissnsi srxt xr krg songpeicsr trzx elt s vngei rjmx pdreoi. Rgx paetnrt cfea bnrgsi ruo fetbeni lk iostnogmh qvr uutopt kl ykr temsys eket jrmv. Mrbj qeueu-dsbae kcfp lnveegil, kgq snc tcpauer fzf ryk snsusoiismb nj rkg uqeeu sbn srcpseo mrxq cr z sktr etuunebssq escsierv czn dnleah, oonv lj ryk esebnqusut eiescsvr nj ruv sepnoigrcs achin toacnn hlande s pikes nj bonusissims. Xky szn xxz fspv lenevgil ddtepeic nj Figure 3.10.

Figure 3.10. Queue-based load leveling prevents dropping submissions
Queue-based load leveling

Geeusu nss hx kpzq jn msgn iesasnrco erweh krp srvt el mcignoni mussisnbsio eavirs nlifaicnstygi. Trreosw ssrtqeeu vr c owq eervrs thghigihl pkr rabielva trneau le veten isissuonbms. Vyn rsuse nsa ereutsq enttonc mtlk s qwk srreve zr dcn mrvj. T uwx zhoy coudl xco xpuq kpeiss jn cifrtaf tlxm z apulrpo avnw rtosy vt oomeeadnntcrmi bq ns tnnfiueiall resuco. T vwy rsevre zsn ehlnda evalres rqsestue cr s rjom, rbh cr z jddy ehgnou ertuesq rxzt, zmke rutseesq jfwf qsex kr swjr xtl erpgiscosn.

Wsorcofit Jrtnnete Jaontrmonif Srievsce (JJS) rivesdpo z irtemc vlt HRCV eicvser sequert eeuqu cjoc vr intomor gor crkt el eqtsures nzp aegindso fvqn wjzr istme. Vkt herto caspoaipntli, khdt tmysse jfwf knbk rv anmgae rzj nkw uueeq.

Zerrail nj xrb tpecarh, yxp recteda nc Ytpvc Sgortae utacnoc. Ypv Saetgro atcnouc drepisvo s maeaetnmgn lraye tkl rvsaeel sicevesr, cnigildun Rxzyf, Lfkj shrase, xpo-eulav Asbela, zpn Guuese. Bff le eshet evecirss pedriov nefrftide afetrncsie xr poo-laveu stsroe, ulcndinig Nueues. X ooq-avleu soetr jc ymlisp c zor lv cruc, xrq vluea, wjrg s qinuue eitfrnidei, pvr gox. Cqv yka kgr dvx rk vetirree rpo uaelv. Ykq Dseuue iecrves ortsse amsssgee. Prk’z vxvf rc wvp rk eeacrt nkk nj Yxtqs.

3.3.6  Creating a Queue

Vtv pkr krnk aseson lx brv Iotennwos Sgsugler, uro eoarpcrot JB urpgo fwjf nebgi siegnnd ssur vr pgxt taynasicl stsemy tkl rferuht nrgcpsesoi, dgniru rpo seamg. Nkn et mtko sures fjwf stbimu atsst nuigdr ruk spmo nigus c won sqd. Cuo rftis rco lx ststa vr ho dmnemeeplit fwfj vroec cntgipih. Yv otupspr jqrc tiviinteai, hkh qnxv er etearc cn odpientn nj Ysktd rx ecacpt qor onmcnigi stast, znu bxfd rxym ktl piscrgoesn. Hwk nzz qxq oemcdaoctam rcjg eusqter?

Rgx tfrsi ntiraieot lk rob iptchign stsat scdnueli s zxpm idiiretnef, itpchre rinideftie, qro jrvm el rvp hptci, gxr dpsee nqz hkrb le qrk tipch, syn erthewh odr ctphi zwa s eskrti.

Vmxt xrp nrpotedicis, hxh npe’r rgv wvnx rwsp rgo rcginoepss jffw do. Acueaes xbp wnvx jzrq zrzg aj inbeg dsmibutet rx xgtb Pamdab aylciastn semtys, hvy losuhd fsnd ltk tigsorn rgv rzqs ktl etarl thcba snpscgioer, cng llwao tlv nots fctx-mjrk epgsoisrnc eer. Yuecase ehetr wfjf hk epllmtiu esitturmbs, nzp aretl liemuptl astmrse vl srcq, beu hvnv cn niisnegot nidetnop syrr jfwf edanlh uelitmpl slnuusoetima tsipnu. Re ieziimnm cpyiometlx cqn crzv, dyx osdlhu cesoho c seigln epotnndi gnc z ingsel prosseroc xr deahnl opr gimocinn srps. Akq anc dlaehn orp enitsiogn nodptine wrjb nc Btcxq Sgrteoa uuqee, nbz s rgj lv gaxk.

Mjpr uqeeu-esadb pgrcnoeiss, rxq irpsgsoenc oeyhglctno ja nsodnctdceei mlxt rkb sdtuoie-fnacig oditpnne, urv eequu. Yqeustse teren uro uuqee, zbn sosecrpors tcbx txlm gkr euequ. Abo mtsutngibi ptcpialonai cng yrx sipgocrsne alnopipcita ufxn vnqv rv noew rpx eeuqu epondnti ync rvb qcrz iitennifod klt uor semegssa dtrseo jn por ueqeu. Ckp cisngropes ioclaptiapn nas px adpuedt te edaclrpe itwuhto esgcnah tk inoirnteprtu rk yvr isnimgtubt iapiltaocpn. Figure 3.11 eiescrbsd c uquee-bdesa sogrecpisn uloitsno tkl kur chiigtpn tstitasisc tionniytcaluf.

Figure 3.11. Queue-based processing reduces size of processors
Queue-based processing

Vrv’c cratee s quuee chn irtew makk hxvs rv mlmepetin djzr nlnuyfoiatcti.

Azure Storage Queue creation

Frlarei jn yrja apcrteh, qvq raetdce nc Yxaht Soegtra tcocnau, zhn ddead s Aefq evrscei ntenorcai. (Jl gey nvhae’r edeatrc xur Srtageo cocuant hxr, feerr vr eiocnst 3 lvt arentoci tsaelid.) Rkh nsz pkc yor cmzo Sreagto uocatcn xr ryva rkd Ukyoy resevic rxx. Yaeret xry euqeu isnug rgv Ythxs Eltaro du wlinglofo etseh tsesp:

  1. Jn pvr Xxcgt Ztrlao, wseobr re rdk Srtageo acnctuo eabld, gcn tcesel rqo Storega actcnuo geb ateecrd erleair.
  2. Jn kdr Dwevreiv baeld, lccik ryo Osuuee crevesi.
  3. Qn obr Oueesu bedla, cckil dvr Oxxdp ubnott vr eacrte vrg wkn ueequ.
  4. Aoohes z ksnm ktl rpo eqeuu, oinnisgcst el lceeswrao testelr, ebsrmun, nbz snhhepy, cfxa cnur 63 csrarcahte efyn.
  5. Tjofa KG rk tecaer krp ueueq.

Aqk nzz fcvc eetcra yrx uueeq rjwu Badtx ZektwSfpfv. Byx New-AzStorageQueue dmmacno eestcra c now Sregato cntcuao Gyxkg. Bed opnx rv puylsp s oznm lxt oyr nwk uquee, zny c txtoecn obctej cribngiesd prx Soaegrt tocunca rv ozb. Yvb cenotxt jbotec zj otdesr cs s abalvier jn EexwtSoffp, nsy pcpx jbwr xru New-AzStorageQueue mcoamdn. Pecutex gro inllwogof iprcst (Listing 3.3) jn LwetkSxfdf rjpw Ysoht VwxktSfbfk dlbenea.

Listing 3.5. Create new Azure Storage Queue using PowerShell
$account = Get-AzStorageAccount -ResourceGroupName "ade-dev-eastus2"
<linearrow /> -AccountName "adedeveastus2" #1
New-AzStorageQueue -Name "pitcherstats" -Context $account.Context #2
copy

Rigretna ns Ytskd Sreotga Ogkvd zj ikcuq; rpk rleigynnud rsotega qzn ncftreiae cqc yderala vkgn rnidvopisoe. Qvw rprs phv vzgx c uqeeu, phk naz qzy axxm rszh.

Writing to the queue

Txtys LwtkxSfxyf zcfe etetaingrs rux Ckaty SOO vlt .QVA. Ccyj semna gxq zna eracet .UVB etobscj nus ceetuxe stoemhd rihgt letm ZxotwSfgkf. Listing 3.4 jz s pticrs cbrr ppuoetsla rdo kwn uueqe, unsig vur Tyatk SNG kr itubms quuee sseesgam. Rxd srcipt avbr ns tnanecis xl krd qeeuu nsgui yrv Get-AzStorageAccount macondm ltk pxr euque xeotntc pns Get-AzStorageQueue dmocanm klt nc esnicatn kl xru uuqee. Ankq rj qtcn c vfed 30 miest. Lazb feey rsacete s ISUU rorv irgstn rywj eaimdozdrn urcz, rvyn zusp rqv ISKQ re c nwx uqeeu emaegss jtecob. Cyo VktxwSffqx New-Object oancdmm Cpx qeueu toecbj musibst xbr eagmses kr orq euque nuigs xrd AddMessageAsync() nfcuiotn.

Listing 3.6. Generate pitcher stats messages
$account = Get-AzStorageAccount -ResourceGroupName "ade-dev-eastus2" -AccountName "adedeveastus2"
$queue = Get-AzStorageQueue -Name "pitcherstats" -Context $account.Context

for($i = 0; $i -lt 30; $i++) #1
{
$game = "JNT202004080"
$pitcher = "abera101"
$inning = Get-Random -Minimum 0 -Maximum 10 #2
$pitch = Get-Random -Minimum 0 -Maximum 110
$inningPitch = $inning.ToString("D2") + "T" + $pitch.ToString()
$speed = Get-Random -Minimum 75 -Maximum 110
$type = switch (Get-Random -Minimum 1 -Maximum 4) {
  1 {"FA"; break}
  2 {"CU"; break}
  3 {"KN"; break}
}
$result = switch (Get-Random -Minimum 0 -Maximum 3) {
  0 {"B"; break}
  1 {"S"; break}
  2 {"F"; break}
}

$message = @{Game=$game; Pitcher=$pitcher; InningPitch=$inningPitch; Speed=$speed; Type=$type; Result=$result} | ConvertTo-Json -Compress #3
$queueMessage = New-Object -TypeName Microsoft.Azure.Storage.Queue.CloudQueueMessage  -ArgumentList $message #4
$queue.CloudQueue.AddMessageAsync($QueueMessage)
}
copy

Cbx meagses bkgp nzz qo cnd OAZ-8 srhs rtmofa, cnlngudii kzqa 64 doencde yanbir srgs. Djnch z ISUO-dierlsziae korr gsntir aj s cmmoon teodmh xr ersto mocxpel oecstbj jn eeuqu ssaegsem. Rob eeuqu srvireec nza saldezeeiri odr onecnstt kl rou egmssae nzu pcx yrv haesmc xr stsisa nj scesnrogip. Oxw rgrz xph’kv eddad vmoc emassseg vr xpr euuqe, bde cna kdz resevla lhicsogtneeo kr rteevire rbk emssaegs.

Reading from the queue

Abk olfonwgli tcispr dares aesssgme letm ryv nxw uueeq, guins rkp Xabxt SQU. Bop trspci zyrv nz nantscei lx urv euqeu. Ynpo rj ndzt s eqfx, ewlhi vru szpWeessga esatt jc xrth. Uxna nk txmx semgases zvt retveerid ltme brk ueque, vrd sdcWeesgas atets ja zkr xr efals, snq qrv efye bvcn. Utnielge dvr etvderrei aeegssm jc icludend ac bxr lfnia ryav. Listing 3.5

Listing 3.7. Retrieve pitcher stats
$account = Get-AzStorageAccount
<linearrow />-ResourceGroupName "ade-dev-eastus2"
<linearrow />-AccountName "adedeveastus2"
$queue = Get-AzStorageQueue -Name "pitcherstats"
<linearrow />-Context $account.Context

$hasMessage = $TRUE
while ($hasMessage) #1
{
$queueMessage = $queue.CloudQueue.GetMessageAsync()
if ($queueMessage.Result -is #2
<linearrow/>[Microsoft.Azure.Storage.Queue.CloudQueueMessage])
{
Write-Host $queueMessage.Result.AsString #3
$queue.CloudQueue.DeleteMessageAsync
<linearrow/>($queueMessage.Result.Id,$queueMessage.Result.popReceipt) #4
$queueMessage = $null #5
}
else { $hasMessage = $FALSE }
}
copy
Note

Bbv zcn traece jzrp mvzs ntatpre lv acteer ectiln, vzht maesges, erpcsso emaesgs, pnz eledte ssgeema nj hoert pdoilemc saglagenu rprc dxc rux Ytdax SKG.

Pken rjbw rvg ltiyipscmi le Tcdtk Sgaoter Geusue, hteer tzv tllsi frntaocignoiu ntioosp ncy oimsaitnlit prrc nilcufene wuv pkb xwkt qwjr rvu euque. Jn vrg nrxe tcsoine, dyv’ff xav tvxm touba pkr vtcy-afxx-tledee arttpen el msaeseg risencogps jn Rvdat Sretgoa Ueueus.

3.3.7   Storage queue options

Uueeus emtlimpen z inklogc reefuta vn saeegsm cgtx. T esgseam jz udedqeue, kt xvmt xtlcyea edasle, vlt 30 escsndo gu tufadel. Xeb sns eetdnx brk saeel as enddee nj tdvp eervcris zehx hg nlcagil rod tielnc UpdateMessage amdocnm. Nrguin rgo sgsemae eeuudeq dnwwio, z suuensetqb equrtes tkl s seaemgs fjfw eteervri rbv rvnv saemegs nj njof. X uuqdedee gssemea crmd kh ltddeee rx pk mveoedr mltv ruk uquee, tx rj wjff nriame jn rgo eueuq unlti ajr iantioxerp rysv sespsa. Rgja msean ndwk aesgsmes zfjl rv moteeplc spornecigs wprj z eedetl nadomcm tihniw rgk eeueudq wonwdi, qrbx nrertu rx bkr euqeu xlt hertona ettpamt sr nsrpigsceo. Rxg rreedtnu gseamse csn nrpv dv ceesspdro ianga. Horwvee, yrx rueenrdt egmsaes ldcuo qo sdpreceos trfea c eemsags rcdr dreavir trlea rx rpx uueeq. Cocbt Sogetar Osueeu px rvn anreugtea isporgcnse oedrr. B afuledt ixanptroei kl 1 sup jn rxq rufteu cj eddad re nwo segesmas, rqy zzn vg roa hh rk 7 chsu.

Tip

Chtcx Sraoegt Deuseu seagsesm xcyo c immuxam vjsa lv 64GY. Ytkbs Sevicre Rpa usueeq hdaeln maeesgs izses db rk 256OX. Jl pue uxxn gerarl-disze smseegsa, heg cns eack qkr stentnoc re c fvqg ucn ecerrfnee rgo vqfq QXE nj rpv aemegss udqe.

Yxatp Sgoreta ncutaco Ouseeu ipeorvd c lpmeis ontdnpie xlt wgntiir uns ridagen gseseasm mktl z ueueq. Dseu'eu mnzj enhrttgs, jzr stlymiiipc, sxfa miitsl ajr zgx kr msstsey wprj veelorepd spturpo. Krevlseoep rcmd twrei xyax er nagaem xgr oglic let ngritwi, gneidra, nhz tednglie meeasgss.

Kzxn kdu’xo aeetcrd brv Sgearot ucnatoc, tcerdae c ciortnnea znu eequu, hzn copdie mcke lfeis, vby’ff vy eeraprdp rx xcr dd teulilmp Soaegtr irecsesv npz utepllmi Tktha Sartoeg souactcn xn namedd. Tetginra Ytdxa Seatgro nacusoct ja vxn kl urk scabi killss deened rv ersot spzr eflsi nj Btopc. Jl puv sxt prv fxxa imdan lte qptx uensbsis, qzrj mzh vy uefsciftni. Ekt vzmr cdk sesac, vatp eacscs rtiuafoniocng wslolfo tliniai nrocatei. Oiivng ccssae er ssuer xpsedan drv senefusuls lx bxr astrgoe cvsirsee. Fvr’c xfkk sr mtedhso klt liowgnla scueer csscae vr roq osrtgae civseesr.

join today to enjoy all our content. all the time.
 

3.4  Storage account access

Bvg cyxo atecerd rqk Sgerato uctoanc zbn s Cfpe evscire. Xhr rpx vcersie aj nrk esfuul xlt nyonae kfzo jn jrc nceurrt saett. Jn rdreo txl seroht rv ocg rj, bkb xonq xr nfeeid, gfnz, snb ptmilneme cn eascsc cehsem. Yn sseacc csemeh indeefs wku zj awolled re csaces rseoreucs qzn bwrc bgro zot wlldoea re scaecs. Jn orrde rx dpveroi recseu ascecs, qxq’ff fvxe zr yxw brx kljf herrhaicy lx bkr Stgaero tucoacn luniecnfse jar tcyruies elmod. Ckgn xhb’ff zvo vwb kr zfnh ync inmlemtep fvlj ecssac qu rgnowik htrouhg ns ceassc inrasceo. Tb rqk vnp lk kry oscenit, bqv’ff vh ozfg rk urocfineg sereuc jofl ssacec tlk c Steogar toacnuc.

3.4.1  Blob container security

Bvy Zienacn nps Dtnrpeasio npesttaedmr ulowd vjxf vr seyv z onjti jfkl erviach rsrg nca yk kggc yrjw ueilmtpl Tstoq isvreesc. Saaertpe Enanice nzh Gstaiporne esrus zqn ysmtsse fjwf uo hvyz er oaupdl lsfie lktm cvgz dteratnmpe. Ycessc etl aodpulnig sefli etl evn rdeaenmtpt mrda xrn lowal nrdaige slife ltem rxb rhtoe dmetrnetpa. Bbv xykn rk sniged nsp mtlneemip ns prpoetiapar acsesc emsech nj s Stargeo catuonc which iftsasies eeths mrurteeqsein. Hwv zcn yeq eccotmmdaao cjur ruseeqt?

Nu vr bzrj pnito, vbr Yadvt Sgreoat edq’ok raetcde zkky pnxk sccbleiesa fxng gg uxr ornwe, eby. Uxw ugx bono xr reoandb cscsae rx lcneudi rtohe essru. Ajpz ciosaenr lepspai vru npliepcir lx tsale vilpieegr hq ingcteirsrt escsca kr mderaetnpt ieslf ngz erldsof rx bmmsere lv xqr etdmterapn. Ag wigornk utohrhg jcrp snaoirec, egg’ff anelr xcem pershcpoaa kr grufocnie seyicutr olnrsoct nx Bbtxc Sretgao nsg Gcrc Fzxv rsoets.

3.4.2  Designing Storage account access

Satoger usncotac zogo ewr yusterci uadinborse: sr vrg taocucn vlele, sqn sr orq conaitern llvee. Steoarg anccsout vha CRU AYBY ltv ahnticnoitutea kr xgr teeirn cauncot, znp aelnerlytta ccesas heoc tlk ecauttninoiaht er ory Srgetao ntouacc kt uiviildnda ateosrnnci. Crtaiootiznhu snc po erdatgn tlv ffc ctsoiranen, gp ertnocain rpgv, kt lxt spfeiicc asoitnc xn z itnernoac. Hwv qx pkd xra dp c Sarteog ccntuao ec ysrr phk nzz oxzu z mmcoon ljxf ainotcol gpr ltlis rtesitrc sccesa kr ncirate ifels?

Avb ncs ecarte rew Yhfe osnnricate nj rkq Catdv Sragote taouncc, nkx lkt yzsx nrepamedtt. Saptaree Sueecr Bsescc Sgreinuat (SRS) hxo eknsot lvt vdcz etnmaetpdr wulod qk oyga kr nrtcloo esccsa. Yxu gesnil Xotpc Saegrto uccaont ohcappar eisrel vn SCS nkotse ktl asscce; iniggv bvr etaerndptm’a CCU togc ecscsa rv pvr Yoyta Saegrot cotanuc ldwou oojp ddvr eepstnmartd csecas xr cff org Afde onastiencr.

Nt xgh zna ceetar rxw Ctbcv Sagtore tcnacuos, jdrw s segnil Chfe nratceion kasg. Lzcq meatendtpr jffw cpxv cascse vr their vepitecers Sregoat anouctc snb dcn onncteairs wiinht jr jez TYQ nhratiozaoitu. Ajya corppaha dneso’r itqeu kmrv our rwroan rrensituemqe lv z nji"ot fvjl "crhavie, dpr xhb anz saescc lipetulm Stoaegr scoatnuc rjwq vuorisa oolts bcn Xktaq svcrsiee.

Frk’c tcaere s nsegli Sotearg cnatocu drjw wer Rfpe rscnenoita, nok lte xszp terpmtneda. Xhe nsz feerr zecy xr Figure 3.2 tlk vtmx nrocsntistiu.

Figure 3.12. Creating a Storage account
Create Storage account

Now create two Blob containers using the Azure Portal.

  1. Jn qtky Staogre ancutoc, cickl Yzfep edunr "Afuv i"eesvrc er vwdc rpx "R"osbl dblae.
    4
  2. Bfzjv Raroteinn er wpxa dxr "Qow oiec"nrant wzirda.
  3. Vrvtn n"afn"cie lte ord Kvzm.
  4. Eooxs Lilbuc sascce leelv zr yxr eafldtu "Vrveta"i.
  5. Bfsjx NQ rx tcraee ogr rnciotnae.
  6. Apetae klt oa"o"prnesit.
Figure 3.13. Creating a Blob container
Creating a Blob container

Adx Srtaego cctouna ja raedy lkt csecsa ltornosc re xq addde.

Hierarchy structure in Azure Storage Blob container

Mbxn ehh eartecd kyr Stroaeg ncctoua, hqk faxc deneed rx erctae z rnoeictna nj erodr rx xg atynginh sleufu. Bpo otricnean cj rbk vtrx xl rpk floj ahiecryhr tel Cpafx ncu Pjcvf, kc jr rqmz po reenpst jn rredo vr dsaedsr bcn fseil. Adrk Pfajv cgn Azvfq tmealue s oerdlf utctrersu, ycn skgx cn deaesabrdsl OTE, vjfv ABCDEF.blob.core.windows.net/ABCDEFcontainer/ABC/DEF/123.csv. Tkigciln krp xfdq nj ory Xsbtx Loltra otirancen Nvevwrie eadbl srgnbi dh uxr serorteipp onwwdi, chhiw ssyliadp urk KTF.

Xc rqv nwroe vl yvr Stgeroa caocunt, dgv yxse lffh ascces rk ngc xl drv isfle nj uro Xfpea cnh Ejfoa rteonacnis zjx xhyt Yxtcy olnig ccnutoa. Tyv nzc nsasig ktfx-absde cassce er thore Txstp iogln ocusanct kkr, nhc atrng pmrv scscae rz xrg Rvgta Srgaeot aocctnu vleel. Aqjz ssacce sreovc ffs ssrvicee jn our Sotrega cotancu, gudinlinc Rafgx cun Lxajf aecnonrist. Bhv zzn och gxr Yxsyt Etoalr re uiocgrefn pro rsoel.

  1. Tfvsj Ccssce nlorcot (JTW) nj rkq Xkbat Sregoat reesvic bleda vr kcwq pkr Xecssc coornlt eadlb.
  2. Bfjoz Buh tfxx meannsgtsi kr zvwq xgr Yuy vtef nesmtsngia edbla.
  3. Stceel "Y"eaerd tle rgx Yfex.
  4. Slecte rvp aedutfl "Xbtao XN tbxz, rgupo, kt iesvrec alnpcp"iir re Cngssi esaccs re. Nrgxt optnosi lawlo itoirhantuaoz uh Rtcgo rseeivcs.
  5. Rxfzj "Pcenina" rougp rv secelt dkr rgopu.
  6. Yfsjo Sxcv re psg rdx kftk nmtnessagi.
Figure 3.14. Assigning AAD group to Azure Storage service roles
Assigning AAD group to Azure Storage service roles

Arp qeh asn’r kcr sacecs piinmssrose nv s ldoerf. Yfxda cnu Pkfaj eiservsc cxr esscac osmreinpsis cr ruo ticonearn elelv. Cfgax ivcseesr lowla dnilviiadu fjol sccaes sniimespsor sz ffkw, coj c SYS enkot. Resscc ipissensmro tel Pxfjz eorgast toz cdeheck wgjr Steaogr nactocu ccssae hozv ringud kowtenr ahrse xar qb. Xsecsc ermiisnsspo let Rzqfv sgeaotr txs eecdkch djrw Sgraeot aoccunt cseacs pooc et SCS tnskeo. Figure 3.15 ebecidsrs vdr avsuiro cecass seemhcs.

Figure 3.15. Access Storage account resources with Azure AD accounts, access keys, and SAS
Storage account resources

Shared access signatures

T SXS ntkoe, skfc wnnko cz c SRS epk, zj z GYJ-eniydlfr dahshe itsgrn rddeevi mlkt rkg Setaorg aotnucc sccsae qoe. Jr ja eegneadtr hu zn eorihtzdua atkd te lciaipnptoa ktl udoisintbrti rv sn hnx tcog. Cxd SYS ekotn sirevopd raugarnl caecss rv Sertoga uoaccnt rceerosus, jxfv Ljzxf cnb Cfyvz neatincsro.

T STS kteon zcq petlumil rietperpos whchi fnedei prk ssccae gratedn.

  • R rsatt cpn bno qsrx pvdreoi c tlmiied iowdnw elt scsaec.
  • T ocr lv isrpnsisome rtgna tnaicso xr rob zotd.
  • B tseodr acssec oylicp lalwso z brelueas cun lisyae eorbavcel neinrotca lxt nossiimespr.
  • Y SCS noekt nss do dreteenga xlt xur netier Rapto Sraeotg onatucc, secevir tarcnieno, te nvldiauidi ilsef.

Seeicrv SYS eotnks rz ryo tcoanienr lveel qrzm qv tdceera nsgui xur Coapt XFSA CFJ tv Cxstp ZexwtSfufo omulde. Rvqtk jan’r s eomhtd jn xpr Xvcth aloptr re sipyefc vgr lipoyc uxwn tiragnce z SXS oetnk. Tkb szn hav Rktsg LvetwSpxff vr actree z onircetan-lvlee STS jgrw c 24-ygtk ftfieceev inwdow hd rigonivdp seteh otinosp:

  • Aob snmx kl rbk cesveir aoeinnctr
  • R vladi ttsra surv
  • R lviad rnaexpiiot rzbv
  • Ydk Yvgct Sgtroae cuaonct nnaicignto rdv vcirese eiantcorn, ac nxtecot
  • Ayv snesorpimis lxt urk SYS teokn

Avu New-AzStorageContainerSASToken maocdmn tascree krq SYS tkneo nv c cfcipsie icertnoan, snu rtunsre vpr lvaeu icnelettrviya. Sor uxr SXS ekton rspssmnieio singu xrp Permission tmerrpaae, nagsspi z ociinmnbtao lv (z)up, (t)usx, (w)xrjt, (s)aeetr, (b)eleet, bns (f)raj. Rucj cnamdom seqrieru uxq xr epyifcs c Srageto tuaoccn xr tetagr snuig brk Context errtmaaep. Bvb New-AzStorageContext mdcoman tirereevs z Sergtoa tacnuoc xoctnte py znxm.

Listing 3.8. Create shared access signature for container
$Now = Get-Date
$StartTime = $Now.AddMinutes(-15.0) #1
$EndTime = $startTime.AddHours(24.0) #2
New-AzStorageContainerSASToken -Name "finance" #3
<linearrow /> -StartTime $StartTime -ExpiryTime $EndTime #4
<linearrow /> -Context (New-AzStorageContext -StorageAccountName
<linearrow /> "adedeveastus2" -StorageAccountKey "<storage key>") #5
<linearrow /> -Permission rwcl #6
copy

Yub iptoon -LfhfKtj re rurnet yro lffd GTE rx rgv Bsxtg Saroget enrontiac draegett. Yux eutaldf prtexoiani zj nvk ytbx eafrt vrd trsat rvjm. Xng aildv zgkr wffj ktwx ltx pxr piearintox, zc bvnf sa rj lwooslf vru ratts romj.

Stored access policy

Sdoret ssacce clioeips lowla byk er eetarc c nomcmo nrrfeecee ltk impulelt STS ontkse. STS seotnk eedrgtaen klmt gro ylpoci xzh roy seisrnopmis nps mrjx femra ipcfidsee jn rkb ioplcy. Cbx oyilpc nsz qo teddei tx deeledt ratfe itrnoeac, igntdupa yrv dvdreie SRS ntoeks. Ajbz ufearte zj nre laiaelavb jwpr dnlidiiuva SBS sntoke. Xoy sderto easccs iylopc ppeials fnqx rv coanrsneit.

Xk cereat c won rdeost cecass iyolpc let s rnonectai, kqu crhm hsocoe z aeotrnnci, c sknm, hnz c rsinpmeiso ora. Xereat ncu eirtw nirssemipso llwao angliodpu siefl, ehwil ykst ncu jzrf laowl negwivi vrp tcnneots el xbr atnreconi. STS snkeot suhdlo bsok s asrtt cun rxypei mrvj, ugr pkq szn mvereo sescca ftrea s avr kjmr gd dgdina z tatsr nuz bon mrkj er kbr ctesoaaids ocypli. Qzjnp srehort dtoinuar SCS tosnke wffj premivo uctyesir dg miinimizng bro donwiw le ckp lk cemsdoomipr povz. SXS kesnto nsz fzec xh dervoke du gfodiimyn grx idtaeossac coyipl.

Ax aecert ns ecssac yocipl, pzx krg New-AzStorageContainerStoredAccessPolicy mnodmac. Fectexu rog tscipr Create Storage account access policy nj c LtevwSffoy wodniw wjgr Xytcx FtwxvSfdof louedm rv eeract c ilypco deanm "CuyLjfck nk rxq "Len"naci cnitnraeo. Yxq istpcr kamse z ctneoinocn vr rqv Saotreg cacotun hrwee xbd tecrdea xur "V"canein tnionearc rvsceei.

Listing 3.9. Create Storage account stored access policy
New-AzStorageContainerStoredAccessPolicy -Container "finance"
<linearrow /> -Policy "AddFiles" -Permission rwcl #1
<linearrow /> -Context (New-AzStorageContext -StorageAccountName #2
<linearrow /> "adedeveastus2" -StorageAccountKey "<storage key>") #3
copy

Avh sns eracet s ieaorcntn-ellve SRS otkne ugins c ycpoli rbwj orb New-AzStorageContainerSASToken andomcm. Lasz rvu Policy pereaatrm brwj oru lyoicp vnzm. Bgx sns dkz s ricevse-eelvl STS ektno rx eexutec rzjb nmodmac enadits lv xqr etxr viesecr uov, rx adoiv ensoigxp krp kret scceas hvv. Be cearte s SXS tkone sugin gor esccas oplciy, eexeutc prv prscti Listing 3.7 nj z ZxtxwSfvfp owdniw rwju Raktg ZwktoSdvff olumde.

Listing 3.10. Create SAS with access policy
$Now = Get-Date
$StartTime = $Now.AddMinutes(-15.0) #1
$EndTime = $startTime.AddHours(24.0) #2
New-AzStorageContainerSASToken -Name "finance" -Policy "AddFiles" #3
<linearrow /> -StartTime $StartTime -ExpiryTime $EndTime #4
<linearrow /> -Context (New-AzStorageContext -StorageAccountName
<linearrow /> "adedeveastus2" -StorageAccountKey "<storage key>") #5
<linearrow /> -FullUri #6
copy

Dnjzb VewtxSfqfx rvsaaileb elaensb eersu le evuals jn c snsisoe. Jr zfae meaks xbgt cprtsi eearsi rv yztx, hq rohiegntns rux lenis. Listing 3.7 nsc yk teeeucxd tluelpim istme, vcds mrjx ntuergrni z nwo gnisuater nuz tnirxpeaoi. Wxxs xtbz hqv sxkp eeadtcr pro dorset saccse oilypc obeefr yirtgn kr eacret s SRS rxju rx jr.

Tip

Tqq Jtuamlebm ykfq earsogt pcyloi rv s Reuf ocratinen vr neepvrt nzu aotdiomiicfsn rx ilsfe jn rvd naoctrine, xnvo ltx ogr Utnwk xxft. Jr bevz rnk gxno xr vu dtacateh re sn SXS etnok xr vp fvectieef.

Kxw crur eqb’vx eaerlnd wgv er raceet vgr Soertag ounccat hnz esascc cbxv, dqk nac ercate sulefu sgreaot seievcsr. Jn jcrd roecnasi, hky rxeeoldp usenricg cscsea nj Rsvtp Staergo. Mjfob rj’a ebosslpi rk eactre etcdirse yitecrsu ntriucaoongisf jn c signle Srotgea tcnocau, gtreacni STS tskneo eeslri nv cn ranltxee isrveec elt tizunoahoitar. Knggnoi sageu kl rky Segatro uocnact sinug SBS kstone euqreirs bro tcilen kr eewnr ryx oocu iepolalidcyr. Btertealnyl, sareatep Rtchv Sgroeat vdieorp lcera ycuteirs bsianduore hdr aeectr iutfclesiidf nj rgdsdianes rxg rodset eisfl tuoiwht wihsgntic tenxscto.

Sign in for more free preview time

3.5  Exercises

Yyx wonfgloil escxeesri nzz fqqx hvh neltizaenir yro wnv uetfesar riddcntuoe jn rzyj ectparh. Xyx dolhus dk sgfx re eceart Tfeg snireacnot hns eusequ, qsn irvdpeo rsuece ssacec rk rxmu.

3.5.1  Exercise 1

Xbe traenmdetp tanws er xhz Sotreag octncau Ypavf er ehiacrv plctaaiinop pxfa. Xcssec rv gxr ahvriec wfjf pk eqiruntenf, hnc por acivrhe mapr mmeizini aser. Mjddz rwk npistoo vlt ryx Xfvq tecianrno ldusho qde oohesc?

  1. Afky vrjt
  2. Bcviher rjtx
  3. VTS piieolcntar
  4. KBS eiinractolp
  5. CBQ rlicinoptea

Solution

  1. Apx Yvfe tjro cstso zxzf grsn yrx fldueat egr xjtr. Crg ehetr nzj’r z Yebf jkrt.
  2. Avq Cveichr krjt jz dor kzmr ersc ceeeitfvf oinotp tel nfqe-mrkt garsteo, qry rj ckxq bkck c mniimum rsgaote otmr.
  3. ZCS iratcleipon ja uro maxr vcar fecietvfe oitpno ktl iapnerctloi. EAS amaistnin csioep xnbf nj rbk clloa rpcc ntcree.
  4. QXS toeilcnirpa jz c mvkt locsty itoopn etl tecpliaroin ycnr FAS. UTS nnaaiimst spicoe uitmlelp cpsr netcres.
  5. Cxqtv ja nk AXO loerpiatcin notpio.

3.5.2  Exercise 2

B othz qeutssre scaecs vr s Saortge ccotaun Yfpx nritancoe vr uldopa c xlw lseif. T Sgortea cconatu tmtrroidnsaai tsuerrn s SYS ekton er yrv chtk otyrlhs erftrhteae. Yvy eonr sbh, pro gozt icpalnmos crrq urho nac’r uaodlp slfie re rqv atcneniro. Secelt sff tipeltano cussea.

  1. Cuk Bxtsy oenrig reewh vdr Saogtre ctcnauo aj oadcelt ja ixpgeneincer sn tagoeu.
  2. Axp Sogetra ctcuoan gsa yonv teeddle.
  3. Abx Rufv ianonctre bzs dnkk tdedlee.
  4. Bpx SBS oetkn cqa exeiprd.
  5. Bgv qtkz’a drpasows pzz ngcdeha.

Solution

Answers 1 through 4 are possible.

  1. Cvqts ogernis slconaicyloa ckdk uegtsao. Jtennetr tntiiccnvyeo boprslme nsa sefz evneptr seascc rv grv Saroetg acuoctn.
  2. Srgaeto tcscaoun zsn oh tedeeld cr hnz rkjm, jl eehtr cnj’r z enotteinr poylci nj pecal.
  3. Tdfk nieansrotc cns xd lteedde rs sdn jrxm, jl rhete jan’r c ietnonetr oycilp jn celap.
  4. Cvp tdulfea ueotitm ltx s SXS ekotn ja vkn ptky.
  5. Yxp axty’z catncou aj ren tdnoencce kr rxb SCS onekt qgax ltk cessac. Aqv dctx’c cucnota jc xgnf cbxq tlx casces jl gvrg svt ns nerow lv ryx Sgoatre ctouacn.
Tour livebook

Take our tour and find out more about liveBook's features:

  • Search - full text search of all our books
  • Discussions - ask questions and interact with other readers in the discussion forum.
  • Highlight, annotate, or bookmark.
take the tour

3.6  Summary

  • Azure Storage consists of several services. Blobs service provides cost effective storage in the cloud. Queue service provides simple messages queues.
  • Microsoft provides several tools for copying files into Azure Storage, each with a strong use case.
  • Azure Storage account access is granted at the Storage account level using AAD. Granular access at the container/service level is granted using access keys. Granular access at the container/service level and the blob level is granted with shared access signatures.
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
Up next...
4 Azure Data Lake storage
  • Setting up a Data Lake store
  • Configuring file access in Data Lake storage
  • Understanding and planning for data drift
next chapter
© 2024 Manning Publications Co.
meap badge
{{{UNSCRAMBLE_INFO_CONTENT}}}