CSC 594 Topics in AI Natural Language Processing

CSC 594 Topics in AI Natural Language Processing

CSC 594 Topics in AI Natural Language Processing Spring 2018 10. Part-Of-Speech Tagging, HMM (1) (Some slides adapted from Jurafsky & Martin, and Raymond Mooney at UT Austin) POS Tagging The process of assigning a part-of-speech or lexical class marker to each word in a sentence (and all sentences in a collection). Input: the lead paint is unsafe Output: the/Det lead/N paint/N is/V unsafe/Adj Speech and Language Processing - Jurafsky and Martin 2 Why is POS Tagging Useful? First step of a vast number of practical tasks Helps in stemming/lemmatization Parsing Need to know if a word is an N or V before you can parse

Parsers can build trees directly on the POS tags instead of maintaining a lexicon Information Extraction Finding names, relations, etc. Machine Translation Selecting words of specific Parts of Speech (e.g. nouns) in pre-processing documents (for IR etc.) Speech and Language Processing - Jurafsky and Martin 3 Parts of Speech 8 (ish) traditional parts of speech Noun, verb, adjective, preposition, adverb, article, interjection, pronoun, conjunction, etc Called: parts-of-speech, lexical categories, word classes, morphological classes, lexical tags... Lots of debate within linguistics about the number, nature, and universality of these Well completely ignore this debate.

Speech and Language Processing - Jurafsky and Martin 4 POS examples N V ADJ ADV P PRO DET noun

chair, bandwidth, pacing verb study, debate, munch adjective purple, tall, ridiculous adverb unfortunately, slowly preposition of, by, to pronoun I, me, mine determiner the, a, that, those Speech and Language Processing - Jurafsky and Martin 5 POS Tagging The process of assigning a part-of-speech or lexical class marker to each word in a collection. WORD the koala put the keys on

P the table tag DET N V DET N DET N Speech and Language Processing - Jurafsky and Martin 6 Why is POS Tagging Useful? First step of a vast number of practical tasks Speech synthesis

How to pronounce lead? INsult inSULT OBject obJECT OVERflow overFLOW DIScount disCOUNT CONtent conTENT Parsing Need to know if a word is an N or V before you can parse Information extraction Finding names, relations, etc. Machine Translation

Speech and Language Processing - Jurafsky and Martin 7 Open and Closed Classes Closed class: a small fixed membership Prepositions: of, in, by, Auxiliaries: may, can, will had, been, Pronouns: I, you, she, mine, his, them, Usually function words (short common words which play a role in grammar) Open class: new ones can be created all the time English has 4: Nouns, Verbs, Adjectives, Adverbs Many languages have these 4, but not all! Speech and Language Processing - Jurafsky and Martin

8 Open Class Words Nouns Proper nouns (Boulder, Granby, Eli Manning) English capitalizes these. Common nouns (the rest). Count nouns and mass nouns Count: have plurals, get counted: goat/goats, one goat, two goats Mass: dont get counted (snow, salt, communism) (*two snows) Adverbs: tend to modify things Unfortunately, John walked home extremely slowly yesterday Directional/locative adverbs (here,home, downhill) Degree adverbs (extremely, very, somewhat) Manner adverbs (slowly, slinkily, delicately)

Verbs In English, have morphological affixes (eat/eats/eaten) Speech and Language Processing - Jurafsky and Martin 9 Closed Class Words Examples: prepositions: on, under, over, particles: up, down, on, off, determiners: a, an, the, pronouns: she, who, I, .. conjunctions: and, but, or, auxiliary verbs: can, may should, numerals: one, two, three, third,

Speech and Language Processing - Jurafsky and Martin 10 Prepositions from CELEX Speech and Language Processing - Jurafsky and Martin 11 English Particles Speech and Language Processing - Jurafsky and Martin 12 Conjunctions Speech and Language Processing - Jurafsky and Martin 13

POS Tagging Choosing a Tagset There are so many parts of speech, potential distinctions we can draw To do POS tagging, we need to choose a standard set of tags to work with Could pick very coarse tagsets N, V, Adj, Adv. More commonly used set is finer grained, the Penn TreeBank tagset, 45 tags PRP$, WRB, WP$, VBG Even more fine-grained tagsets exist Speech and Language Processing - Jurafsky and Martin 14 Penn TreeBank POS Tagset Speech and Language Processing - Jurafsky and Martin

15 Using the Penn Tagset The/DT grand/JJ jury/NN commented/VBD on/IN a/DT number/NN of/IN other/JJ topics/NNS ./. Prepositions and subordinating conjunctions marked IN (although/IN I/PRP..) Except the preposition/complementizer to is just marked TO. Speech and Language Processing - Jurafsky and Martin 16 POS Tagging Words often have more than one POS: back The back door = JJ On my back = NN

Win the voters back = RB Promised to back the bill = VB The POS tagging problem is to determine the POS tag for a particular instance of a word. Speech and Language Processing - Jurafsky and Martin 17 How Hard is POS Tagging? Measuring Ambiguity Speech and Language Processing - Jurafsky and Martin 18 Two Methods for POS Tagging 1. Rule-based tagging 2. Stochastic 1. Probabilistic sequence models

HMM (Hidden Markov Model) tagging MEMMs (Maximum Entropy Markov Models) Speech and Language Processing - Jurafsky and Martin 19 POS Tagging as Sequence Classification We are given a sentence (an observation or sequence of observations) Secretariat is expected to race tomorrow What is the best sequence of tags that corresponds to this sequence of observations? Probabilistic view Consider all possible sequences of tags Out of this universe of sequences, choose the tag sequence which is most probable given the observation sequence of n words w1wn. Speech and Language Processing - Jurafsky and Martin

20 Classification Learning Typical machine learning addresses the problem of classifying a feature-vector description into a fixed number of classes. There are many standard learning methods for this task: Decision Trees and Rule Learning Nave Bayes and Bayesian Networks Logistic Regression / Maximum Entropy (MaxEnt) Perceptron and Neural Networks Support Vector Machines (SVMs) Nearest-Neighbor / Instance-Based Raymond Mooney (UT Austin)

21 21 Beyond Classification Learning Standard classification problem assumes individual cases are disconnected and independent (i.i.d.: independently and identically distributed). Many NLP problems do not satisfy this assumption and involve making many connected decisions, each resolving a different ambiguity, but which are mutually dependent. More sophisticated learning and inference techniques are needed to handle such situations in general. Raymond Mooney (UT Austin) 22 Sequence Labeling Problem Many NLP problems can viewed as sequence labeling. Each token in a sequence is assigned a label.

Labels of tokens are dependent on the labels of other tokens in the sequence, particularly their neighbors (not i.i.d). foo bar blam zonk zonk Raymond Mooney (UT Austin) bar blam 23 Information Extraction Identify phrases in language that refer to specific

types of entities and relations in text. Named entity recognition is task of identifying names of people, places, organizations, etc. in text. people organizations places Michael Dell is the CEO of Dell Computer Corporation and lives in Austin Texas. Extract pieces of information relevant to a specific application, e.g. used car ads: make model year mileage price For sale, 2002 Toyota Prius, 20,000 mi, $15K or best offer. Available starting July 30, 2006. Raymond Mooney (UT Austin) 24 Semantic Role Labeling For each clause, determine the semantic role played by each noun phrase that is an argument to the verb. agent patient source destination instrument John drove Mary from Austin to Dallas in his Toyota Prius. The hammer broke the window.

Also referred to a case role analysis, thematic analysis, and shallow semantic parsing Raymond Mooney (UT Austin) 25 Bioinformatics Sequence labeling also valuable in labeling genetic sequences in genome analysis. extron intron AGCTAACGTTCGATACGGATTACAGCCT Raymond Mooney (UT Austin) 26 Problems with Sequence Labeling as Classification Not easy to integrate information from category of tokens on both sides. Difficult to propagate uncertainty between decisions and

collectively determine the most likely joint assignment of categories to all of the tokens in a sequence. Raymond Mooney (UT Austin) 27 Probabilistic Sequence Models Probabilistic sequence models allow integrating uncertainty over multiple, interdependent classifications and collectively determine the most likely global assignment. Two standard models Hidden Markov Model (HMM) Conditional Random Field (CRF) Raymond Mooney (UT Austin) 28 Markov Model / Markov Chain A finite state machine with probabilistic state transitions. Makes Markov assumption that next state only depends

on the current state and independent of previous history. Raymond Mooney (UT Austin) 29 Getting to HMMs We want, out of all sequences of n tags t1tn the single tag sequence such that P(t1tn|w1wn) is highest. Hat ^ means our estimate of the best one Argmaxx f(x) means the x such that f(x) is maximized Speech and Language Processing - Jurafsky and Martin 30 Getting to HMMs This equation should give us the best tag sequence But how to make it operational? How to compute

this value? Intuition of Bayesian inference: Use Bayes rule to transform this equation into a set of probabilities that are easier to compute (and give the right answer) Speech and Language Processing - Jurafsky and Martin 31 Using Bayes Rule Know this. Speech and Language Processing - Jurafsky and Martin 32 Likelihood and Prior Speech and Language Processing - Jurafsky and Martin 33

Two Kinds of Probabilities 1. Tag transition probabilities -- p(ti|ti-1) Determiners likely to precede adjs and nouns That/DT flight/NN The/DT yellow/JJ hat/NN So we expect P(NN|DT) and P(JJ|DT) to be high Compute P(NN|DT) by counting in a labeled corpus: Speech and Language Processing - Jurafsky and Martin 34 Two Kinds of Probabilities 2. Word likelihood/emission probabilities p(wi|ti) VBZ (3sg Pres Verb) likely to be is Compute P(is|VBZ) by counting in a labeled corpus: Speech and Language Processing - Jurafsky and Martin 35 Example: The Verb race

Secretariat/NNP is/VBZ expected/VBN to/TO race/VB tomorrow/NR People/NNS continue/VB to/TO inquire/VB the/DT reason/NN for/IN the/DT race/NN for/IN outer/JJ space/NN How do we pick the right tag? Speech and Language Processing - Jurafsky and Martin 36 Disambiguating race Speech and Language Processing - Jurafsky and Martin 37 Disambiguating race Speech and Language Processing - Jurafsky and Martin 38

Example P(NN|TO) = .00047 P(VB|TO) = .83 P(race|NN) = .00057 P(race|VB) = .00012 P(NR|VB) = .0027 P(NR|NN) = .0012 P(VB|TO)P(NR|VB)P(race|VB) = .00000027 P(NN|TO)P(NR|NN)P(race|NN)=.00000000032 So we (correctly) choose the verb tag for race Speech and Language Processing - Jurafsky and Martin 39 Hidden Markov Models

What weve just described is called a Hidden Markov Model (HMM) This is a kind of generative model. There is a hidden underlying generator of observable events The hidden generator can be modeled as a network of states and transitions We want to infer the underlying state sequence given the observed event sequence Speech and Language Processing - Jurafsky and Martin 40 Hidden Markov Models States Q = q1, q2qN; Observations O= o1, o2oN; Each observation is a symbol from a vocabulary V = {v1,v2,vV} Transition probabilities Transition

probability matrix A = {a } aij =P(q t = j | qt- 1 =i) 1 i, ijj N Observation likelihoods Output probability matrix B={bi(k)} b (k) =P(X =o | q =i) i t k t Special initial probability vector =P(q =i) 1 i N i 1 Speech and Language Processing - Jurafsky and Martin

41 HMMs for Ice Cream You are a climatologist in the year 2799 studying global warming You cant find any records of the weather in Baltimore for summer of 2007 But you find Jason Eisners diary which lists how many ice-creams Jason ate every day that summer Your job: figure out how hot it was each day Speech and Language Processing - Jurafsky and Martin 42 Eisner Task Given Ice Cream Observation Sequence: 1,2,3,2,2,2,3 Produce: Hidden Weather Sequence: H,C,H,H,H,C, C

Speech and Language Processing - Jurafsky and Martin 43 HMM for Ice Cream Speech and Language Processing - Jurafsky and Martin 44 Ice Cream HMM Lets just do 131 as the sequence How many underlying state (hot/cold) sequences are there? HHH HHC HCH HCC CCC CCH CHC CHH

Argmax P(sequence | 1 3 1) How do you pick the right one? Speech and Language Processing - Jurafsky and Martin 45 Ice Cream HMM Lets just do 1 sequence: CHC Cold as the initial state P(Cold|Start) Observing a 1 on a cold day P(1 | Cold) Hot as the next state P(Hot | Cold) Observing a 3 on a hot day P(3 | Hot) Cold as the next state P(Cold|Hot) Observing a 1 on a cold day P(1 | Cold)

.2 .5 .4 .4 .3 .0024 .5 Speech and Language Processing - Jurafsky and Martin 46 POS Transition Probabilities Speech and Language Processing - Jurafsky and Martin 47 Observation Likelihoods Speech and Language Processing - Jurafsky and Martin 48

Question If there are 30 or so tags in the Penn set And the average sentence is around 20 words... How many tag sequences do we have to enumerate to argmax over in the worst case scenario? 3020 Speech and Language Processing - Jurafsky and Martin 49 Three Problems Given this framework there are 3 problems that we can pose to an HMM Given an observation sequence, what is the probability of that sequence given a model? Given an observation sequence and a model, what is the most likely state sequence? Given an observation sequence, find the best model parameters for a partially specified model

Speech and Language Processing - Jurafsky and Martin 50 Problem 1: Obserbation Likelihood The probability of a sequence given a model... Used in model development... How do I know if some change I made to the model is making things better? And in classification tasks Word spotting in ASR, language identification, speaker identification, author identification, etc. Train one HMM model per class Given an observation, pass it to each model and compute P(seq|model). Speech and Language Processing - Jurafsky and Martin 51

Problem 2: Decoding Most probable state sequence given a model and an observation sequence Typically used in tagging problems, where the tags correspond to hidden states As well see almost any problem can be cast as a sequence labeling problem Speech and Language Processing - Jurafsky and Martin 52 Problem 3: Learning Infer the best model parameters, given a partial model and an observation sequence... That is, fill in the A and B tables with the right numbers... The numbers that make the observation sequence most likely Useful for getting an HMM without having to hire

annotators... That is, you tell me how many tags there are and give me a boatload of untagged text, and I can give you back a part of speech tagger. Speech and Language Processing - Jurafsky and Martin 53 Solutions Problem 2: Viterbi Problem 1: Forward Problem 3: Forward-Backward An instance of EM Speech and Language Processing - Jurafsky and Martin 54 Problem 2: Decoding Ok, assume we have a complete model that can give us what we need. Recall that we need to get

We could just enumerate all paths (as we did with the ice cream example) given the input and use the model to assign probabilities to each. Not a good idea. Luckily dynamic programming helps us here Speech and Language Processing - Jurafsky and Martin 55 Intuition Consider a state sequence (tag sequence) that ends at some state j (i.e., has a particular tag T at the end) The probability of that tag sequence can be broken into parts The probability of the BEST tag sequence up through j-1 Multiplied by the transition probability from the tag at the end of the j-1 sequence to T. And the observation probability of the observed word given tag T

Speech and Language Processing - Jurafsky and Martin 56 Viterbi Algorithm Create an array Columns corresponding to observations Rows corresponding to possible hidden states Recursively compute the probability of the most likely subsequence of states that accounts for the first t observations and ends in state sj. vt ( j ) = max P(q0 , q1 ,..., qt - 1 , o1 ,..., ot , qt =s j | ) q0 , q1 ,..., qt - 1 Also record backpointers that subsequently allow backtracing the most probable state sequence. Speech and Language Processing - Jurafsky and Martin 57

Computing the Viterbi Scores Initialization v1 ( j ) =a0 j b j (o1 ) 1 j N Recursion N vt ( j ) =max vt - 1 (i )aijb j (ot ) 1 j N , 1 t T i =1 Termination N P* =vT 1 ( s F ) =max vT (i )aiF i =1 Raymond Mooney at UT Austin 58 58

Viterbi Backpointers s1 s2 s0

sN t1 t2 t3 Raymond Mooney at UT Austin

tT-1 tT sF 59 Viterbi Backtrace s1 s2

s0 sN t1

t2 t3 tT-1 tT

sF Most likely Sequence: s0 sN s1 s2 s2 sF Raymond Mooney at UT Austin 60 60 The Viterbi Algorithm Speech and Language Processing - Jurafsky and Martin 61 Viterbi Example (1): Ice Cream Speech and Language Processing - Jurafsky and Martin 62 Viterbi Example (1) Viterbi example: Ice Cream

A: Transition Probability Matrix p(|C) p(C|) 0.5 p(H|) 0.4 p(STOP|) 0.1 p(|H) p(1|) 0.5 p(2|) 0.4 p(3|) 0.1 p(|START) 0.3 0.2 0.6 0.8 0.1

0 B: Emission Probablity Matrix p(|C) Path Probability Matrix for 3 1 3 p(|H) 0.2 0.4 3 Cold 0.02 Hot 0.32 1 3 0.048 0.0024

0.0384 0.009216 1 3 end 0.00024 0.000922 Backpointer Matrix 3 Cold start Hot start <-- Hot <-- Hot end <-- Cold <-- Cold

<-- Hot <-- Hot 0.4 * The Termination step obtains the largest probability of 0.0009216 for the state Hot. This is the probability for the whole sequence. * Back-tracing yields the re-constructed path of [Hot, Hot, Hot]. 63 Viterbi Summary Create an array With columns corresponding to inputs Rows corresponding to possible states Sweep through the array in one pass filling the columns left to right using our transition probs and observations probs Dynamic programming key is that we need only store the MAX prob path to each cell, (not all paths). Speech and Language Processing - Jurafsky and Martin

64 Evaluation So once you have you POS tagger running how do you evaluate it? Overall error rate with respect to a gold-standard test set. Error rates on particular tags Error rates on particular words Tag confusions... Speech and Language Processing - Jurafsky and Martin 65 Error Analysis Look at a confusion matrix

See what errors are causing problems Noun (NN) vs ProperNoun (NNP) vs Adj (JJ) Preterite (VBD) vs Participle (VBN) vs Adjective (JJ) Speech and Language Processing - Jurafsky and Martin 66 Evaluation The result is compared with a manually coded Gold Standard Typically accuracy reaches 96-97% This may be compared with result for a baseline tagger (one that uses no context). Important: 100% is impossible even for human annotators. Speech and Language Processing - Jurafsky and Martin 67 Viterbi Example (2)

Fish sleep. Ralph Grishman at NYU 68 A Simple POS HMM 0.2 start 0.8 noun 0.1 0.8 verb 0.7

end 0.2 0.1 0.1 Ralph Grishman at NYU 69 Word Emission Probabilities P ( word | state ) A two-word language: fish and sleep Suppose in our training corpus, fish appears 8 times as a noun and 5 times as a verb sleep appears twice as a noun and 5 times as a verb Emission probabilities: Noun P(fish | noun) : 0.8 P(sleep | noun) : 0.2

Verb P(fish | verb) : 0.5 P(sleep | verb) : 0.5 Ralph Grishman at NYU 70 Viterbi Probabilities 0 1 2 3 start verb noun end

Ralph Grishman at NYU 71 0.2 start 0.8 noun 0.1 0.8 0.7 verb end 0.2 0.1

0.1 0 start 1 verb 0 noun 0 end 0 1 Ralph Grishman at NYU

2 3 72 0.2 start 0.8 noun 0.1 0.8 0.7 verb end

0.2 0.1 0.1 Token 1: fish 0 1 start 1 0 verb 0 .2 * .5

noun 0 .8 * .8 end 0 0 Ralph Grishman at NYU 2 3 73 0.2 start 0.8

noun 0.1 0.8 0.7 verb end 0.2 0.1 0.1 Token 1: fish 0 1

start 1 0 verb 0 .1 noun 0 .64 end 0 0

Ralph Grishman at NYU 2 3 74 0.2 start 0.8 noun 0.1 0.8 0.7 verb end

0.2 0.1 0.1 Token 2: sleep 0 1 2 start 1 0 0 verb

0 .1 .1*.1*.5 noun 0 .64 .1*.2*.2 end 0 0 -

(if fish is verb) Ralph Grishman at NYU 3 75 0.2 start 0.8 noun 0.1 0.8 0.7 verb end

0.2 0.1 0.1 Token 2: sleep 0 1 2 start 1 0 0 verb

0 .1 .005 noun 0 .64 .004 end 0 0 -

(if fish is verb) Ralph Grishman at NYU 3 76 0.2 start 0.8 noun 0.1 0.8 0.7 verb end

0.2 0.1 0.1 Token 2: sleep (if fish is a noun) 0 1 2 start 1 0 0

verb 0 .1 noun 0 .64 .005 .64*.8*.5 .004 .64*.1*.2 end 0 0 Ralph Grishman at NYU

3 77 0.2 start 0.8 noun 0.1 0.8 0.7 verb end 0.2 0.1

0.1 Token 2: sleep (if fish is a noun) 0 1 2 start 1 0 0 verb 0

.1 noun 0 .64 .005 .256 .004 .0128 end 0 0 Ralph Grishman at NYU 3

78 0.2 start 0.8 noun 0.1 0.8 0.7 verb end 0.2 0.1 Token 2: sleep take maximum, set back pointers

0.1 0 1 2 start 1 0 0 verb 0 .1

noun 0 .64 .005 .256 .004 .0128 end 0 0 Ralph Grishman at NYU 3 79 0.2

start 0.8 noun 0.1 0.8 0.7 verb end 0.2 0.1 Token 2: sleep take maximum, set back pointers 0.1

0 1 2 start 1 0 0 verb 0 .1 .256 noun

0 .64 .0128 end 0 0 - Ralph Grishman at NYU 3 80 0.2 start

0.8 noun 0.1 0.8 end 0.7 verb 0.2 0.1 0.1 Token 3: end 0

1 2 start 1 0 0 verb 0 .1 .256 - noun 0

.64 .0128 - end 0 0 Ralph Grishman at NYU - 3 0 .256*.7 .0128*.1 81 0.2 start

0.8 noun 0.1 0.8 end 0.7 verb 0.2 0.1 Token 3: end take maximum, set back pointers 0.1 0

1 2 start 1 0 0 verb 0 .1 .256 - noun

0 .64 .0128 - end 0 0 Ralph Grishman at NYU - 3 0 .256*.7 .0128*.1 82 0.2

start 0.8 noun 0.1 0.8 end 0.7 verb 0.2 0.1 Decode: fish = noun sleep = verb 0.1

0 1 2 start 1 0 0 verb 0 .1 .256 - noun

0 .64 .0128 - end 0 0 Ralph Grishman at NYU - 3 0 .256*.7 83 Complexity?

How does time for Viterbi search depend on number of states and number of words? Ralph Grishman at NYU 84 Complexity time = O ( s2 n) for s states and n words (Relatively fast: for 40 states and 20 words, 32,000 steps) Ralph Grishman at NYU 85 Problem 1: Forward Given an observation sequence return the probability of the sequence given the model... Well in a normal Markov model, the states and the sequences are identical... So the probability of a sequence is the probability of the path sequence

But not in an HMM... Remember that any number of sequences might be responsible for any given observation sequence. Speech and Language Processing - Jurafsky and Martin 86 Forward Efficiently computes the probability of an observed sequence given a model P(sequence|model) Nearly identical to Viterbi; replace the MAX with a SUM Speech and Language Processing - Jurafsky and Martin 87 Ice Cream Example Speech and Language Processing - Jurafsky and Martin

88 Ice Cream Example Speech and Language Processing - Jurafsky and Martin 89 Forward Speech and Language Processing - Jurafsky and Martin 90

Recently Viewed Presentations

  • Macbeth context revision Today we will:  Know why

    Macbeth context revision Today we will: Know why

    Macbeth written 1606 . Shortly before Macbeth was written James became king of England. He was really into the supernatural and had written a book called 'Demonology' about witches and how to catch them. Shakespeare needed to impress him to...
  • Practicas de Compensaciones en Contextos Inflacionarios

    Practicas de Compensaciones en Contextos Inflacionarios

    Intereses profundos no son ni los hobbies ni los entusiasmos en algunos temas . son las pasiones, modelos de identificación y valores internalizados. Los intereses de vida definen que trabajo una persona ama. Esa meta debería ser la que nos...
  • WELCOME TO YOUR WELL-BEING PROGRAM Blue Cross &

    WELCOME TO YOUR WELL-BEING PROGRAM Blue Cross &

    Blue Cross & Blue Shield of Rhode Island brings you an effective and enjoyable program, powered by Virgin Pulse®. The more you engage with it, the better you can feel, and the more you'll be rewarded! *This program is available...
  • PROYECTO - Mis clases en UNITEC

    PROYECTO - Mis clases en UNITEC

    Premisas comunes de las terapias construccionistas posmodernas (Harlene Anderson, 2004) Crítica hacia la noción de verdad absoluta, universal y objetivas. El conocimiento y las realidades son construidas socialmente. A través del lenguaje las personas atribuyen significados a su mundo y...
  • Phrasal Verbs / Verbes prépositionnels

    Phrasal Verbs / Verbes prépositionnels

    PHRASAL VERBS IN THE PRESS. Duringnumeroustakes, Daniel Radcliffe sings to himself, tapping out a complexrhythm on the « dragon's » neck; psychinghimself up for the nextjump by jigging about ».
  • Mean share of winner - Political Studies Association

    Mean share of winner - Political Studies Association

    Local Elections in England, Scotland and Wales 2017. PSA Media Briefing - 25 April 2017. ... Erosion of Lib Dem local base shown in reduced by-election contestation but now growing. ... [email protected]
  • Food insecurity: How to monitor a complex problem

    Food insecurity: How to monitor a complex problem

    Choosing the weights for aggregating the selected indicators into a composite index. Use of preference elicitation techniques to determine the weights that reflect the relative importance of the identified dimensions. Current activities at FAO.
  • Risk Assessment of Maritime Navigation across the Greater

    Risk Assessment of Maritime Navigation across the Greater

    Safety of Life at Sea (SOLAS) Convention, requires contracting governments to provide navigation safety services. MARPOL Convention - preventing . and minimizing pollution from ships - both accidental pollution and that from routine operations ...