Distributed Computing & Middleware

Distributed Computing & Middleware

Distributed Computing & Middleware Info on CptS 464 Good background in case you never take 464 Introduction A distributed system is one in which hardware or software components located at networked computers communicate and coordinate their actions only by message passing Very broad definition Lots of examples Lots of kinds Abbreviations Distributed System by DS, Distributed Computing is DC

You know you have one when the crash of a computer youve never heard of stops you from getting any work done. Leslie Lamport Example Local Call Caller: // declare and init stuff x = new int [100]; y = new util; // create and rtn ref flag = y.sort(x, 100); Callee: // declare and init stuff (not shown) int util:sort(int [] a, int max) { // implementation of sort... bubble sort, quicksort, return status; } Example Local Call (BACKUP SLIDE) Caller: // declare and init stuff

x = new int [100]; y = new util; flag = y.sort(x, 100); Callee: // declare and init stuff int util:sort(int [] a, int max) { // implementation of sort... return status; } Potential assumptions:

Object Invocation conventions between caller (client) and callee In same address space (on same computer) In same programming language (usually) Written by same programmer (often, not always) Same operating system for both caller and callee Same CPU type for both caller and callee Can transfer data and control quickly, effectively in zero time Both fail, or neither do (for the most part) None of these assumptions are always true in a distributed system! Reminder: Assembler Example C-like call X = 4 + ((Y * 4) / (A + B)); Equivalent assembler (vars on stack)

ldr r1, [sp, Y] !load Y mul r1, r1, #4 !Y * 4 ldr r2, [sp, A] !load A ldr r3, [sp, B] !load B add r2, r2, r3 !A + B div r1, r1, r2 !divide the two add r1, r1, #4 !add four to result str r1, [sp, X] !store result in ! X on stack Reminder: Calling Conventions To call a function or routine you need to push arguments on the stack (in the right order), push the return address on the stack, .., branch to the routine, . Calling conventions define this for a given compiler/language High-level language compilers do all this for you Have to program yourself if using assembler Calling myFunc() in C: int main() {

int x = 1; int y = 2; int z = myFunc(x,y); } int myFunc(int x, int y) { return x + y } Reminder: Calling Conventions myFunc: movl movl add movl ret %edi, %esi, %esi,

%esi, -4(%rbp) -8(%rbp) %edi %eax .globl main main: movl $1, -4(%rbp) movl $2, -8(%rbp) call ret myFunc !grab x off stack !grab y off stack !add x and y !return x + y

!x = 1 !y = 2 Example Local Call (2) Potential assumptions between caller and callee: Assembler calling conventions In same address space (on same computer) In same programming language (usually) Same operating system Same CPU type Can transfer data and control quickly, effectively in zero time Both fail, or neither do (for the most part) None of these assumptions are always true in a distributed system! Example Remote Call Caller: // declare and init stuff x = new int [100]; Y = new util.lookup();

Flag = y.sort(x, 100); // proxy or stub // generated by middleware int util:sort(int [] a, int max){ // put a[], max into struct // send message with struct // wait: message w/ struct // copy from struct to a[], // status return status; } Callee: // declare and init stuff int util_impl:sort(int[] a, int max){ // implementation of sort return status; } // skeleton generated

// by middleware compiler // receive message with struct // copy from struct to a[], max flag = z.sort(a, max) // copy a[], flag into struct // send message with struct Many Local Call Assumptions dont Hold! Not a local object Invocation, so need more help Need remote equivalent of local (assembler) calling conventions In this class we will come to understand this plumbing much better Not in same programming language (cant assume) Not written by same programmer Not running same operating system for caller and callee Not same CPU type for caller and callee

Many Local Call Assumptions dont Hold! (2) Not always in the same administrative domain Latency for transfer of control and data can be large and, worse, unpredictable Partial failures Membership of the system (the computers in its collection) can change Unreliable or insecure communication Context: (Most) Technology Marches On Hardware technologys progress phenomenal in last few decades Moores Law Metcalfs Law Graphics processing power Software technologys progress is much more spotty

Software crisis Yet SW is a large and increasing part of complex apps/systems! Apps and systems are rapidly becoming (more) networked Oops, distributed software is much harder yet to get right Middleware a promising technology for programmability of distributed systems CptS 464/564 12 Middleware in Context: 2011 David E. Bakken Why Middleware? Middleware == A layer of software above the operating system but below the application program that provides a common programming abstraction across a distributed system

Middleware exists to help manage the complexity and heterogeneity inherent in distributed systems Middleware provides higher-level building blocks (abstractions) for programmers than the OS provides Can make code much more portable Can make them much more productive Can make the resulting code have fewer errors Analogy MW:sockets HOL:assembler Middleware sometimes is informally called plumbing Connects parts of a distributed application with data pipes and passes data between them CptS 464/564 13 Middleware in Context: 2011 David E. Bakken Middleware in Context Host 1 Distributed Application

Host 2 Distributed Application Client Middleware API Middleware API Middleware Middleware Operating System API OS Comm. Processing Server Operating System API

OS Storage Network Middleware in Context: 2011 David E. Bakken CptS 464/564 14 Comm. Processing Storage Middleware Benefit: Masking Heterogeneity Middlewares programming building blocks mask heterogeneity Makes programmers life much easier!! Kinds of heterogeneity masked by middleware (MW) frameworks

All MW masks heterogeneity in network technology All MW masks heterogeneity in host CPU Almost all MW masks heterogeneity in operating system (or family thereof) Notable exception: Microsoft middleware (de facto; not de jure or de fiat) Almost all MW masks heterogeneity in programming language Noteable exception: Java RMI Some MW masks heterogeneity in vendor implementations Object Management Group (omg.org) best here: CORBA (objectoriented), DDS (publish-subscribe) CptS 464/564 15 Middleware in Context: 2011 David E. Bakken Middleware Benefit: Transparency Middleware can provide useful transparencies: Access Transparency

Location transparency Concurrency transparency Replication transparency Failure transparency Mobility transparency Masking heterogeneity and providing transparency makes programming distributed systems much easier to do! CptS 464/564 16 Middleware in Context: 2011 David E. Bakken Middleware Perspective Middleware is like underwear: it is absolutely essential, but it should never be seen in public. unknown witticist Background info (only first page required): http://www.eecs.wsu.edu/~bakken/middleware.pdf

17 Context: (Most) Technology Marches On Hardware technologys progress phenomenal in last few decades Moores Law Metcalfs Law Graphics processing power Software technologys progress is much more spotty Software crisis Yet SW is a large and increasing part of complex apps/systems! Apps and systems are rapidly becoming (more) networked Oops, distributed software is much harder yet to get right Middleware a promising technology for programmability

of distributed systems CptS 464/564 18 Middleware in Context: 2011 David E. Bakken Why Middleware? Middleware == A layer of software above the operating system but below the application program that provides a common programming abstraction across a distributed system Middleware exists to help manage the complexity and heterogeneity inherent in distributed systems Middleware provides higher-level building blocks (abstractions) for programmers than the OS provides Can make code much more portable Can make them much more productive Can make the resulting code have fewer errors Analogy MW:sockets HOL:assembler Middleware sometimes is informally called plumbing

Connects parts of a distributed application with data pipes and passes data between them CptS 464/564 19 Middleware in Context: 2011 David E. Bakken Middleware in Context Host 1 Distributed Application Host 2 Distributed Application Client Middleware API

Middleware API Middleware Middleware Operating System API OS Comm. Processing Server Operating System API OS Storage Network Middleware in Context: 2011 David E. Bakken CptS 464/564 20

Comm. Processing Storage Middleware Benefit: Masking Heterogeneity Middlewares programming building blocks mask heterogeneity Makes programmers life much easier!! Kinds of heterogeneity masked by middleware (MW) frameworks All MW masks heterogeneity in network technology All MW masks heterogeneity in host CPU Almost all MW masks heterogeneity in operating system (or family thereof) Notable exception: Microsoft middleware (de facto; not de jure or de fiat) Almost all MW masks heterogeneity in programming language

Noteable exception: Java RMI Some MW masks heterogeneity in vendor implementations Object Management Group (omg.org) best here: CORBA (objectoriented), DDS (publish-subscribe) CptS 464/564 21 Middleware in Context: 2011 David E. Bakken Middleware Benefit: Transparency Middleware can provide useful transparencies: Access Transparency Location transparency Concurrency transparency Replication transparency Failure transparency Mobility transparency Masking heterogeneity and providing transparency makes programming distributed systems much easier to do!

CptS 464/564 22 Middleware in Context: 2011 David E. Bakken Middleware and Legacy Systems Legacy systems are a huge problem (and asset) in industry and military domains! Middleware often called a glue technology: integrated legacy components Much distributed programming involves integrating components, not building them from scratch! Middlewares abstractions are general enough to allow legacy systems to be wrapped Distributed objects are best here because more general End result: a very high-level lowest common denominator of interoperability

23 Middleware vs. Sockets Middleware is much easier to program! Example interface from CORBA (OMG) IDL: module HelloApp { interface Hello { bool MyFunction(in float a, in string b, in int c, in string d, in float e, out double ret); }; }; 24 Middleware vs. Sockets(2) Calling that interface in C++ with CORBA helloImpl (classname) = CORBA_NAME(..) boolean success =

helloImpl.MyFunction(3.3, hello, 2345, bakken!, 67.34, doubleBox); 25 Middleware vs. Sockets (3) float a; char b[5]; int c;char d[7]; float e; double rval; int success //Ignore read errors. Hardcode field size, assume //all systems are same CPU arch. and bit size read(socket, &a, sizeof(float)); read(socket, b, sizeof(char) * 5); read(socket, &c, sizeof(int)); read(socket, d, sizeof(char)*7); read(socket, &e, sizeof(float)); // continued on next slide 26 Middleware vs. Sockets (4)

// continued from previous slide calculating return values etc goes here //send back return value write(socket, &rval, sizeof(double)); //cant tell if it actually was recieved, or if socket is broken write(socket, &success, sizeof(int)); //again, no error checking 27 Middleware vs. Sockets (5) This socket code ignored all of the following: Errors with the socket Differences in CPU architecture (endianness) Differences in representation of data types between languages I/O errors Type checking of data variables

All of the above (and much more) are handled by middleware Middlewares programming building blocks (abstractions) mask heterogeneity Makes programmers life much easier!! 28

Recently Viewed Presentations

  • What is FFA? - rowan.k12.ky.us

    What is FFA? - rowan.k12.ky.us

    Terms Active Agriculture Education/FFA Alumni Leslie Appelgate Collegiate Eagle Ear of Corn FFA Motto Henry Groseclose Honorary C. H. Lane Owl Carlton Patton Plow Program of Activities Rising sun Smith-Hughes Act E. M. Tiffany Explain how, when, and why the...
  • Discussion Questions: 1) Why is it important to

    Discussion Questions: 1) Why is it important to

    Why is it important to understand how coverage of the Olympic Games is evolving from a sports marketing perspective? Who do you think the information from this info graphic might benefit? What are broadcast rights? Why do you think TV...
  • ENCOUNTERING CONFLICT - Ms Rogers' Resources

    ENCOUNTERING CONFLICT - Ms Rogers' Resources

    Coherence and organisation - does my essay have a clear contention; have I used TEEL in my paragraphs? Maturity, sophistication and insight . Key assessment expectations. Students must indicate the text and topic on which their response is based.
  • PowerPoint 演示文稿

    PowerPoint 演示文稿

    Hamlet, Shakespeare. Act 1, Scene 1 ... Once upon a midnight dreary, while I pondered, weak and weary, Over many a quaint and curious volume of forgotten lore— ...
  • Grammar Notes and Mini-Lessons

    Grammar Notes and Mini-Lessons

    Grammar Notes and Mini-Lessons. Warm-up. Complete Exercise A, #1-5 on page 264. ... In ninth grade, we read the story The Most Dangerous Game out of our anthology, Glencoe Literature: The Reader's Choice. ... which he flew from New York...
  • Fire Safety/Extinguisher Use

    Fire Safety/Extinguisher Use

    Together, they produce the CHEMICAL REACTION that we call . FIRE. Fuel Classifications. Fires are classified according to the type of fuel that is burning. If you use the wrong type of fire extinguisher on the wrong class of fire,...
  • Layers of the Earth - Mrs. Tosh's class

    Layers of the Earth - Mrs. Tosh's class

    Crust Layers of Earth Crust Basalt: Ocean crust made of dark dense rock and is very thin Granite: Continental crust that is less dense but is thicker (especially where mountains are present) Oceanic Crust Continental Crust 2-3 miles thick Basalt...
  • Units of measurement Different units of measurement are

    Units of measurement Different units of measurement are

    For example this clock shows the big and little hand pointing to 4 so it is 20 past 4 in the afternoon. If it was 20 to 4 in the afternoon, the little hand would be nearly pointing at the...