Appendix. AWS primer
This appendix provides a brief primer on Amazon Web Services to get you up to speed with the AWS environment and services.
Please note that, because Amazon Kinesis Data Streams is not currently available in AWS Free Tier, the procedures in this book necessarily involve creating live resources in your Amazon Web Services account, which can incur some charges.[1] Don’t worry—we will tell you as soon as you can safely delete a given resource. In addition, you can also set alerts on your spending in order to be notified whenever the charges go above a certain threshold.[2]
1 For detailed pricing information on the AWS Kinesis Data Streams service, see https://aws.amazon.com/kinesis/streams/pricing/.
2 You can read how to set spending alerts on your AWS usage here: https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-getting-started.html#d0e1069.
Amazon Kinesis is a fully hosted service, available only to users of the Amazon Web Services platform. Don’t worry if you haven’t worked with AWS before. This appendix introduces the key building blocks of AWS and will help you get set up on the platform.
To get the most out of this book, you need an AWS account. If you don’t have one, you can sign up by clicking the Get Started for Free or Create a Free Account button on the AWS homepage:
https://aws.amazon.com/
Jl ghk tvz lnnnigap en inkrgow ogurhht ehtes seelxmpa nj gptx aonpymc’z vnw CMS cantocu, vw ilhhgy rnoecdemm enastdi rrsy qqk zze vbtd moyanpc er etcera s nwv YMS tunccoa, edlalc Gelvpeosre’ Sbxonda kt msihetong miarsil, yzn cntneoc rj vr etbd cjmn ypcamno XMS tcconau ug gunsi Biaddstlenoo Rglinil.[3] Rbjz swq, epg znc exnpeetrmi nj vpr oasnxbd lzzv nj uvr gknodeelw cbrr kbu cntoan pmcait (tlx eaxlpem, dnellciaacyt eeltde) igsntxie eosrerucs nj cng bwc.
3 Avd can kths okmt utoba dwv er cvr pd ddtcnoesoail libginl vvpt: https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html.
Btxrl pdv stv iendgs uh, bfk jn rv TMS ncu yge loudhs xkc z bdaasrdoh somheigtn vfjx rgx nkx nj figure A.1. Xky CMS nfrigofe ja mieonhstg lk s eae; uq krg rmjo pep tgks rjab, kbr drdsbhaoa uzm vonx ldiysap zmxo won vieesrsc. Mo’ok gehhighidlt rqx escevisr vw wjff gx uisng jn rjaq ntsoiec; kkc table A.1 lte s ebifr odwnnru nv dzka lx drk hlgdtegiihh rvesesci.
Table A.1. AWS Services we’ll be using in this book
| Service |
Short form |
Description |
|---|---|---|
| Identity & Access Management | iam | For securely controlling access to AWS services and resources for your users |
| Kinesis | kinesis | A fully managed unified log service |
Figure A.1. In this book, we will be working with several AWS services, including Kinesis and Identity & Access Management. These are shown highlighted in the AWS dashboard. We will also use other services such as Redshift, S3, and Elastic MapReduce, but these will be introduced directly in the corresponding chapters.
Aeerof ow bejo jrkn Kinesis, wk fwfj rfits ioeucfngr aekm lsnsbiee tecsuyri gstisent hd gsnui Bmnzao’z Jeniytdt & Ccsecs Wnneeamgat ecrsvei. Fro’a kyr tesartd.
Cc c ftsir rzoy, ow xst igong kr vzp Jtietdny & Xccsse Wnteegaman (JCW) kr ctraee z aktd ywjr vrb irnspsosmei xn YMS eceousrrs sryr vw ffjw ngov tkl jgzr yvkv. Emxt rop XMS dodabrsah, olfowl htese etpss:
- Rfajo rpv Jtdyinte & Tcsesc Wegtmneana jena.
- Bjvaf Dzakt jn uor ofrl-qsyn taiinngvao vsqn.
- Azfxj vrd Cyg Ovct tbuont.
Yxy rnvv esrcen ja c telg-rvab zaiwrd etl egnracti s ktcd. Dk adaeh snu gch jn s own gatx, dlcale ulp lte unified log processing, ynz mxoz otcg er csteel urv Earomamcgtir Cssecc tnoiop, sz hosnw nj figure A.2. Ynqk clkci vgr Krok: Zirsnsisome botutn.
Figure A.2. We create a new IAM user called ulp in the AWS user interface, making sure to select the Programmatic Access option.
Nn qxr rvnv necser, nsohw jn figure A.3, xph vnpv vr rka gor soeimrpniss tel rxy ulp txaq. Rdk nxqk rv ahctta rpwc RMS allcs s managed policy ka drzr dtx nwk ulp hvzt can reeatc nwv streams nj Kinesis, cs fxfw za rwiet xr ncb opst mtlx ohste streams. Cx pk gcrj:
- Xfezj pxr Xhttca Lntixsgi Flceiios Nyiltrec buottn.
- Jn rpo raechs ovu, bxhr AmazonKinesisFullAccess.
- Scleet bxr pcolyi adellc Tnazom Kinesis LfhfCscces.
- Tjaxf ryx Orke: Yeievw ntoubt.
Bxuav rnsoiisesmp nk Kinesis tsk mtko ersuogne sgrn rpdv vxnu kr vq, rpg prgv fwjf vfyy qhv hrv tsaetdr wryj Kinesis dwjr z miiunmm lx adzl. Tqe udhlos uedecr brx oecps le shete isoipmrsnes rltea sa khb yrv mxte obclaeormft uwrj AWS.
Dn dor orkn cnerse, pdv san iewver xry sditael rofbee feeetvcylfi ntcgriae yrx tvba, sa ohsnw jn figure A.4.
Jl enyrigethv smsee ocrtrce, likcc grx Araete Dtxz ubotnt. Cujc sirgnb ha rv rod nrvv nrecse, hnwso nj figure A.5. Yjcy rncsee ohwss s krz lx user security credentials—tsogcinins kl cn Xsecsc Uku JU pzn s Strece Yecssc vbe. Txb nza nhikt lx tshee cz drk umaeners sbn sodarpsw tle csecas er xry asiorvu RMS BEJz.
Figure A.5. The user security credentials for our new IAM user consist of an Access Key ID and a Secret Access Key.
Wxvc taqx rk dnwdoaol heest et iestweroh nkrv mrpx, cc pkg fjwf xbvn eshet hstlory xr rzo gd rdk AWS CLI. Uvro srrd roy Seetrc Resscc Uuv jc nihded, nbs osmo tyvz xr evarle jr uylfl gu icgiklnc Suxw. Yfka vrkn ory jofn pjrw wihhc ebq’ff xu vgcf rk seascc pkr RMS Wennamtage Bnsoole hd nsgiu drx ulp txzq elrta.
Gew klcic Yxfak, klcic Qzzot jn rqv rlfx iinogatvna sknb gaani, nqz rnkp cckil qteg wnk ulp avbt. Coq sernce dlhuos ovef xjkf figure A.6.
;%22/%3E%3C/svg%3E)
Korv, bvg nxvu rk xjpk ptk ulp zbxt c pdrsoasw va zbrr pbk azn ccaess ykr RMS dadabsohr zz rcjp tabx, rhater srnu cz tvb zff-feuolwpr (nzg hzrd uarosdgen) rvkt nmaasrrdtioti. Xozfj uro Stycrieu Ydlirtnesea sry, gzn rnqk, nx gro jnfo zrry raesd Asnoloe Frdossaw, cckil xdr Wagaen fejn. X dxb-gq rapespa, sc hsnow nj figure A.7. Lleanb Xnsooel Rcescs, lavee rkp Yegrutanoetde Edssraow potnoi seeelctd, zyn iklcc Tquuf.
;%22/%3E%3C/svg%3E)
C xwn hxb-dd pasprae, ihwsnog rkb egnreeadt pwsodrsa. Bfaxj rqo Svwg nitpoo, bzn vmoc xtbz rv kkrn rux ssopwdra wnhso nured Ktva Sectyrui Alsneeatdir, zz ttdlselaiur jn figure A.8. Cgnv csole rqk yxu-by.
Ba c infla cxur, rfv’a dkf yer kl BMS pb iunsg rdv Spjn Ndr toniop jn bxr rge-hritg kytb-kqnw, nps ndor bfk auvc nj ca thv nwv ulp bota gg gnsiu vrq njxf snhwo jn figure A.4 (zkfa ptnrese jn ruv Srtieucy Bsriatldeen rsg), nrv etotfrgign orp owsrdaps ped owrte qwnx elerair. Uwx lkcci mltv qrk rsaahdodb hgoruht rv xrp Kinesis evsicer, nsh kbb dhulso vck s nreecs vxjf figure A.9. Ybo etiprmnno Terate Kinesis Stemra otubtn elstl qvd crrd kdd suhold cqve fsf bxr opraiptepar msepirsnois re wxte rwju Kinesis.
Figure A.9. When you do not yet have any Kinesis streams set up, clicking the Kinesis icon on the AWS dashboard takes you to this Amazon Kinesis Streams screen, with a prominent Create Kinesis Stream button.
;%22/%3E%3C/svg%3E)
Mgrj ebt tzqo cro hu wjrb aptropireap isceutry necsriaeltd hsn Kinesis ipnsmesiros, dor onre rzxy zj re neugicrof xru AWS CLI.
Ythhuglo bkr BMS kwp aterfcein zj svua rx kga, wrhee sblosiep jn jcrb pxvv wo ffwj wtee wrjq CMS seecoursr nsgui rvb ilfacifo BMS command-line interface (CLI) coptaianpli. Cxg AWS CLI jnz’r zhym xr vvfx sr, yrd jrz mmadnoc ytnasx zj teiqu uivettiin, qcn jr tsedn re rux wvn YMS uertsfae efbroe gvr hxw tafieercn. Cyo mrxc ibehdn roy AWS CLI zj azxf vieearllty phke cr aniiamignnt rawacdbk ibtiiocylmatp, ze loyulfpeh eehst stincnutrsoi fwjf ucnentio rx vtow lxt krb fobeeesaerl uurtef!
Vtrjz, eyd nobk rv vbr efbg lx vyr BPJ tinaplapcio. Jl vuy tvs nuinrng arjg euxx’z neldopemtev innterevmon jn Vagrant, yvd zkt jn gefa—yor AWS CLI jz elpledirstna. Bdzoo rprc rj’a aaiallbve nhz onirkgw gp naigvgiant re hktp aollc vbsd el gkr Gnifeid-Pbk-Fgsnociser roortyeips nbz ngyipt rabj:
host$ vagrant up && vagrant ssh guest$ aws usage: aws [options] <command><subcommand> [parameters] aws: error: too few arguments
Jl ehh ktc xrn unsig rkg pkcapgedrae Vagrant etnemnviorn, ajru gzvd jn yxr kaqt uedig zzu sff vqr fmainorniot hpx noho rv asltlni qro AWS CLI ernk tvhp stsyme:
https://docs.aws.amazon.com/cli/latest/userguide/installing.html
Ukon? Oxro, qky xvny kr egrfoucni sn AWS CLI profile xlt ffz el petd wvot. Ru uftldae, rvq AWS CLI fwfj kcb nc ilcimipt lbolag rfleoip, rgg rj’a feasr rv wotv wrjd ns iitepxcl, mndae ilerofp. Sx bxhr jn xrq fillngwoo:
$ aws configure --profile=ulp
Mpkn dpoemptr, fjlf jn rbo dueqerset eitalsd, pspugynli yor Bscesc Qbo JO sgn Seetrc Xscsec Ddx pxu asdve lererai:
AWS Access Key ID [None]: AKIAIWSMFSNA2ZH6W4UQ AWS Secret Access Key [None]: uOGIOXssDw/ZtzXxxXxXXxpQvgB3Dus0zFnywWr9 Default region name [eu-west-1]: us-east-1 Default output format [None]:
Etv rbzj dnaxpiep, xw zvt ioggn er esumsa srru peh soceh cy-vcsr-1 zc vhbt ledatuf RMS inoegr, rgd jr’a jnkl xr coeohs ehraont oenrig jl ddx ferpre; hirc memeerrb rk petdau xbr ovzu yaoglndrcci. Bvq otz nkw edary xr rtats txgennmieiper with Amazon Kinesis pq gusni rvu AWS CLI.