2 Overview John Shawe-Taylor, Steve Gunn, Bill Triggs,
2 Overview John Shawe-Taylor, Steve Gunn, Bill Triggs, Sami Kaski, David Barber, Claudio Gentile, Mikio Braun, Jose Balcazar, Michle Sebag, Nicola Cancedda, Mitja Jermol, Alfons Juan, Petri Myllymaki, Nicol Cesa-Bianchi PASCAL2 (FP7-IST-NoE) Pattern Analysis, Statistical Modeling and Computational Learning PASCAL2 Overview Mission statement PASCAL2 will enable a refocusing of the Distributed Institute created in PASCAL towards the emerging challenges created by the ever expanding applications of adaptive systems technology and their central role in the development of large scale cognitive systems. PASCAL2 Overview
Core Challenges Complexity in prior knowledge: language, stochastic differential equation models, etc Learning in multi-component systems: agent learning, cognitive systems, etc Partial or delayed feedback: semi-supervised learning, reinforcement learning, exploration vs. exploitation, etc. Representation and architecture: multimodal processing, symbolic vs subsymbolic, probabilistic inference Scalability and roll-out of adaptive systems technology: plug and play solutions scaling to very large data processing tasks, embedded software development
PASCAL2 Overview Management Structure Strategic Advisory Council Steering Committee executive body meeting twice yearly coordinators, programme managers, gender issues rep, finance and executive officers (~20 people). Executive Committee Executive Officer (UCL), Finance Officer (Soton), Coordinators Individual Programme Committees/Managers Assembly one representative from each site
PASCAL2 Overview Programme Managers Programme Manager Co-Manager Knowledge Management Steve Gunn, Soton-ECS John Shawe-Taylor, UCL Scientific Management John Shawe-Taylor, UCL Steve Gunn, Soton-ECS
Operational Management Steve Gunn, Soton-ECS John Shawe-Taylor, UCL Balance & Integration Dunja Mladenic, JSI Bill Triggs, CNRS Site Activity Bill Triggs, CNRS Chris Williams,
Sami Kaski, HUT PASCAL2 Overview Peter Grunwald, CWI Programme Managers Programme Manager Internal Visiting Claudio Gentile, UniInsubria Conference & Workshop Attendance Petri Myllymaki, UH
External Visitor Nicolo Cesa-Bianchi, UMIl Curriculum Development Colin de la Higuera, CNRS Jose Balcazar, UPC Liaison Florence dAlch, CNRS Marko Grobelnik, JSI Brokerage of Expertise & Public Outreach
David Barber UCL Nello Cristianini, UoB Industrial Club, IPR and Exploitation Marko Grobelnik, JSI Nicola Cancedda, XEROX SAS Pump priming Dunja Mladeni JSI Nicol Cesa-Bianchi Milan PASCAL2 Overview
Balance & Integration Site Activity monitoring Theme Management Joint Research Pump-Priming Harvest Challenges Conference & Workshop Organisation Mobility Internal Visiting Conference & Workshop Attendance External Visitor Outreach Curriculum Development Liaison Brokerage of Expertise & Public Outreach Industrial Club, IPR & Exploitation
Knowledge 4 All Foundation WP2 WP3 WP4 WP5 PASCAL2 Overview Funding/activities 33% of funding for site allocation, remainder for programme activities Second, third, fourth and fifth year site allocation was decided by normal procedure
Funding implementation for non-B members is working smoothly Delayed transition to Beneficiary status for five Non-beneficiaries has been finalised Percentage of funding across the programmes has remained unchanged, but some programmes reduced in final year PASCAL2 Overview Breakdown of Programme budget WP Programme WP1 WP2 Knowledge Management 6.0
Conference & Workshop Attendance 2.0 External Visitor 1.0 Curriculum Development 0.5 Liaison Programme 0.5 Brokerage of Expertise & Public Outreach 1.0 Industrial Club, IPR & Exploitation
0.5 Site Activity Funding Reserve Total PASCAL2 Overview 33.0 2.0 100.0 PASCAL Yr 2 highlights Establishment of a PASCAL web portal, compute cluster,
virtual meeting, and Source Forge collaborative facilities Repository of technical reports with ~3000 entries ~15 Pump-priming research projects ~25 challenges, eg VOC, RTE, Explor. vs exploit. Establishment of a brokerage of expertise web portal ~120 Workshops and summer schools on different project topics focussed around thematic programmes ~ 1000 Recorded lectures from many PASCAL workshops, tutorials, summer schools (http://videolectures.net/pascal) Draft curriculum developed for machine learning, Summer Schools and Boot Camp for new researchers PASCAL - Some Statistics PASCAL 1 56 Contractors + associate members 4+ year project: 20032008 Funding was 6M (inc. Swiss contribution) Dual-band funding model: site funding and programmes PASCAL 2 Initially 40 members now 57 members 5 year project: 20082013 Funding: 6M
Same funding model PASCAL2 Yr 2 Membership Beneficiary Members Non-beneficiary Members 17 partners (cf PASCAL core sites) now ~20 Sign full contract and Consortium Agreement Initially 23 members now increased to 36 Institution signs a Membership Agreement
Industrial Club Members Sign lighter version of Membership Agreement Key interaction through the Harvest Programme Site Funding Site grants are grassroots funding to allow teams to participate actively and achieve network goals An aid to collaboration, integration... Not a payment for excellence / general purpose funding... Allows distributed management and reduces
the number of small requests to programme coordinators Site Funding - Principles Reward a broad spectrum of activities Evaluate many activity classes Avoid narrow focus, extreme scores, arbitrary thresholds or cut-offs Automate the evaluation as much as possible, but allow reviewers some discretion at site level E.g. to correct for forseen changes in site activities: funding is for future activities, not a
reward for past ones. Headline Achievements PASCAL2 Overview Headline Achievements Workshop organisation, challenges and pump-priming activity has continued in line with experiences from first four years with continued submission of publications to the PASCAL repository. Harvest programme sponsored to promote easy access to Videolectures, co-sponsored by Knowledge 4 All Foundation: La Vie (Learning Adapted Video Information Enhancer)
Harvest project to extend the functionality of the PASCAL data repository was funded Challenges have continued to be organised through to the end of the network including one to be reported at the final review. Joint meeting with EUCOGIII as final review meeting of PASCAL2 PASCAL2 Overview Headline Achievements Network has continued to have a dominant presence at top ML conferences Thematic programmes have in some cases set the agenda for international research focus: eg Partial or delayed feedback: bandit algorithms, reinforcement learning, exploration vs. exploitation, etc. Challenges have had a disproportionate influence on a range of research communities: eg VOC, RTE, GREAT.
PASCAL2 Overview Headline Achievements Pump priming has pump primed new research directions: eg CompLACS CWOC has influenced international conference landscape, eg NIPS in Europe, 50% of NIPS workshops supported Videolectures/K4All has established a unique portal for academic videos that is driving the development of online learning provision with recognition from UNESCO Harvest programme has created systems that have impact, eg PronTO and La Vie PASCAL2 Overview Headline Achievements
Organisation of one of the key machine learning conferences ICML2012 at a PASCAL site. Organisation of the premier European Conference in Machine Learning ECML/PKDD 2012 at a PASCAL site. Videolectures (and K4All) received the UNESCO World Summit Award for the best e-Science and Technology product of the decade. Continued development of Knowledge 4 All Foundation including organisation of two events: Large scale online learning and decision making with ComPLACS project and industrial sponsors Co-creation of Emerging Trends in Academia aimed at understanding how technology can affect education in the age of online resources and MOOCs. PASCAL2 Overview
PASCAL Impact There has been a major initiative to assess the impact of the PASCAL investment. This is included in the Final report and will be summarised in Brokerage presentation. Some findings include: Percentage of papers at NIPS including a PASCAL author increased from 15% in 2002 to a maximum of 35%, but maintaining at over 25% on average. Percentage of workshops supported by PASCAL increased from about 20% in 2002 to consistently being more than 50% during PASCAL2 PASCAL2 Overview PASCAL Impact
K4All organised an industry sponsored workshop on Large-Scale Online Learning and Decision Marking in September 2012. high profile speakers/companies involved in the large scale deployment of machine learning, including Microsoft (Thore Graepel, Manik Varma, Sham Kakade, Leon Bottou, John Langford), Netflix (Carlos Gomez-Uribe), Facebook (Ralf Herbrich), Xbox Live (Ulrich Paquet), Causata (Leonard Newnham).
PASCAL2 Overview Operational Management Programme Steve Gunn PASCAL2 Overview Operational Management Programme Monthly meetings of the Executive Committee to coordinate operational matters Procedures for matching itemised billing entries to draft Form Cs has worked well for the first three year finances. Request reporting annually with indication of closure or carry
forward. Itemised billing and funding requests localised by year to enable accurate budgeting and recycling of unused funding. Remaining budget estimated and re-allocated in Steering Committee meetings in final year. Conservative approach to allow for exchange rate fluctuations. PASCAL2 Overview Programme Requests Request Request 0 Accept Reviewers
Programme Manager Reviewer Coordinator Administrator Administrator Site Manager/ Site Site Administrator Administrator Final Final Report Report 8 5 Accept 7 7 Completed
For beneficiaries this will be actionable by the site manager and one other appointed site administrator (could be a researcher or a local administrator) For non-beneficiary sites this will be actionable by the site administrator at UCL. Scientific and Thematic Programme Management John PASCAL2 Overview Cognitive Architecture and Robotics Thematic programme on Cognitive Architectures and Robotics organised European Workshop on Reinforcement Learning 2012 has been supported by the thematic programme
workshop for IROS 2012 was successfully supported by the thematic program with cofunding from EUCOG-III. workshop at NIPS 2012 on Multi-Trade-offs in Machine Learning Continued collaboration with CompLACS project linking ML with robotics platform and news analysis platform developed by PASCAL PASCAL2 Overview Cognitive Inference and Neuroimaging 2nd International Workshop on Pattern Recognition in Neuroimaging (PRNI) that was initiated by the TP: over 100 people with proceedings published by IEEE Toolkit PronTo (developed by a Harvest programme project) team have continued to run training events and extend the s/w NIPS workshop on Machine Learning and Interpretation in NeuroImaging organised at NIPS 2012 Both workshops to be continued
PASCAL2 Overview Activity recognition: a machine learning approach iStethoscope challenge was hosted at AISTATs 2012 Segmenting heart data Classifying heart data into four categories: funding used to pay doctors to do this mobile computing data collection to create large datasets for activity analysis
Baseline system created Data collection underway PASCAL2 Overview Machine learning for multimodal interaction Translectures project funded from Nov 11: La Vie (Learning Adapted Video Information Enhancer) Harvest project co-funded by Translectures
Automatic subtitling of videos Involving Knowledge 4 All Proof of concept recommendation system that provide users with advice on suitable videos for their needs based on ml techniques ICGI 2012 (International Conference on Grammatical Inference) organised by TP PASCAL2 Overview Machine learning for multimodal interaction ICMI Multimodal Grand Challenge organised as part of the International Conference on Multimodal Interaction (ICMI 2012) Final Challenge co-organised with Translectures including Knowledge 4 All Foundation for completion at final meeting: IASD: Interactive Annotation of Sequential Data
PASCAL2 Overview Workshop Organisation Programme Sami PASCAL2 Overview Workshop programme Manager: Sami Kaski, Aalto University Mission: Help fund workshops Processes:
About core machine learning themes To connect to the outside world Core workshop: 2 reviewers, 2500-4000 Non-core: Program manager decides, 2500 Steering: Mostly bottom-up, with quality control by refereeing A few planned bigger events Additional steering by Thematic programmes PASCAL2 Overview Workshop programme Year Five
Two calls for workshops plus running call In total still 39 workshops PASCAL2 Overview Workshop programme Sample workshops European Workshop on Reinforcement Learning ECML/PKDD Workshop on Community Mining and People Recommenders Large-Scale Online Learning and Decision-Making Workshop
19th Machine Learning Summer School International Workshop on Pattern Recognition in Neuroimaging (PRNI) that was initiated by PASCAL but will continue PASCAL2 Overview Workshop programme Highlights in 2012: Three of the main conferences, AISTATS, ICML and ECMLPKDD were organized by PASCAL sites Three summer schools, one co-located with AISTATS
Strong presence at NIPS was continued (7 workshops) PASCAL2 Overview Pump-priming Report Dunja Mladeni, Nicol Cesa-Bianchi PASCAL2 Overview PASCAL2 Pump-priming Overview Call I: end of May 2008, 5 proposals, 3 accepted Budget: 135 025 EUR, Y1 (20%), Y2 (40%), Y3: rest Presentations at PASCAL2 Workshop, September 2009 Call II: mid June 2009, 6 proposals, 2 accepted
Budget: 96 000 , Y2 (20%), Y3: (20%), Y4: (35%), Y5: rest Presentations at PASCAL2 Workshop, March 2012 Call III: mid June 2010, 12 proposals, 3 accepted Budget: 149 402 (after reducing 81k EUR -> 61k EUR) Presentations planned for March or September 2012 No calls in 2011 Total budget distributed: 380 527k PASCAL2 Overview Pump-priming projects Call I http://videolectures.net/ecmlpkdd09_pascal_steering/ (pascal_meet2009) Modelling Learning and Co-Learning Curves for Application to Cognitive-Interfaces, 1.10.2008 31.03.2010 -> 28.2.2011 https://patterns.enm.bris.ac.uk/projects/co-learning
Learning and Inference in Structured, Higher Dimensional Stochastic Differential Systems, 1.1.2009 31.08.2009 -> 1.12.2010 Total: 27175 , Concluded publication Learning the Preferences of News Readers with SVM and Lasso Ranking, AIAI-2010 conference Total: 43 350 , Concluded a joint grant submission to EPSRC on Learning and Inference in Stochastic Differential Systems applied to Epilepsy publication Particle smoothing in continuous time: a fast approach via density estimation, journal IEEE Transactions in signal processing
Context Models for Textual Entailment and their Application to Statistical Machine Translation, 1.11.200831.01.2010->31.12.2010 http://u.cs.biu.ac.il/~nlp/te-smt/ Total: 64 500 , Concluded Publications at ACL-2009 conference, EAMT 2010 conference, Workshop on Machine PASCAL2 OverviewTranslation and Morphologically-rich Languages 2011 Pump-priming projects Call II Sparse Reinforcement Learning in High Dimensions, 1.10.2009 30.09.2012, 45 000 , https://sites.google.com/site/sparserl/
9 conference papers (3 NIPS papers 2009 and 2010, IEEE Allerton 2010, IEEE CDC 2010, ECMLPKDD 2010, EWRL 2011, ICML 2011, AISTAT 2012) Feature selection in Reinforcement Learning, mainly using a sparsity inducing penalty l1 Compressed Sensing for improving linear bandit algorithms when the unknown parameter is assumed to be sparse The project extended due to delays in a partner joning PASCAL2 Semi-supervised learning of semantic spatial concepts for a mobile robot, 1.1.2010 31.12.2012, 46 100 http://homes.dsi.unimi.it/~orabona/, 2 papers (CVPR 2010, ICML 2011) Online algorithms for multiclass multikenel learning Improved algorithms and analysis for online semi-supervised learning The project extended due to difficulties with hiring personnel
PASCAL2 Overview Pump-priming projects Call III Data-Dependent Geometries and Structures : Analyses and Algorithms for Machine Learning, 1.10.2010 30.09.2012 Bridging machine classification and human cognition: discriminative vs. generative learning, 1.7.2011 31.8.2012
Total: 45 000 , http://www.csml.ucl.ac.uk/projects/ddgas/ 6 papers (NIPS-2010, NIPS-2011, DISCML 2011, AISTAT 2012, 2 journal sub.) distribution dependent PAC-Bayes priors, efficiently algorithms (data dependent kernels, acyclic Markov Random Fields), fundamental inequalities, node and link classification on graphs Total: 43 402 , http://ahsu.psychol.ucl.ac.uk/d-MCMCP/ 3 paper submissions structure of the category representations is important -> developing discreteMarkov Chain Monte Carlo with People (d-MCMCP) that uses MCMC sampling to examine the structure of human category representations over discrete objects using d-MCMCP to understand how surgeons classify shoulder fractures, people's choice of hospitals, consumer preferences, moral representations, and facial emotions Online Learning for Robot Control, 1.11.2010 31.12.2012 Total: 61 000 robotics data produced, designing and analyzing online algorithms PASCAL2 Overview
Infrastructure Mikio Braun PASCAL2 Overview mldata - Overview Community website for the exchange of machine learning data sets funded by Pascal2. Organized into Data Sets: Raw data sets, raw or in a number of supported file formats Learning Tasks: Focus on supervised learning
Learning Methods, Results: To track progress Challenges: Collection of learning tasks PASCAL2 Overview mldata landing page PASCAL2 Overview mldata data sets view PASCAL2 Overview mldata - Statistics 843 data sets, 30 learning tasks. Hand populated data sets. 191 data sets from libsvm.
673 data sets from the Weka repository. About 130k downloads in the last year, 150 per data set on average. Most active ones are well-known data sets like the Iris, breast cancer, leukemia, Translation Initiation Site Pred, and MNIST data sets. PASCAL2 Overview mldata - outlook Supports DOI (unique data set ids) minting. Interoperability with other Open Source Projects OpenML (Pascal2 Harvest)
Scikitlearn, one of the important Python machine learning toolboxes Challenge organization within mldata. Indefinite hosting provided by TU Berlin. PASCAL2 Overview Balance and Integration Dunja Mladeni, Bill Triggs PASCAL2 Overview B&I Activity Budget usage: Y1: 6000 , Y2: 8500 , Y3: 2000 , Y4/5: 9000 Y4 no submission transfer the budget to Y5 Activities in Y5
Encouraging women in PASCAL2 to provide their profiles to be publicly available on the Web site link from http://www.k4all.org/ (3 000 EUR) Interviews with female researchers for videolectures.Net invite women in PASCAL and some outside PASCAL @ ECLM PKDD 2012 (500 EUR) Challenges and rewards of a career in research in various different cultures, few women in ICT research, real and perceived glass ceilings, work-life balance One-on-one interviews and a small (3-4 people) group informal discussions http://videolectures.net/womenscience2012_bristol/ PASCAL2 Overview Site Funding Bill Triggs PASCAL2 Overview Liaison Programme
Florence dAlch PASCAL2 Overview Liaison programme Budget : 5000 euros Actions related to Cognitive Science and Robotics community - pursued the actions with EUCOGIII : special session with PASCAL2 At the 3rd EUCOGIII members conference, April 10-11, 2013 at Palma, important delegation of PASCAL2 researchers Jan Peters (Pascal2) invited speaker: - Workshop: Beyond Robot Grasping - Modern Approaches for Learning Dynamic Manipulation organized in Vilamoura, Portugal by Heni Ben Amor,Ashutosh Saxema, Oliver Kroemer and Jan Peters (Pascal2) jointly to the international conference IROS 2012 (Workshop programme) PASCAL2 Overview Liaison programme Actions related to other communities Statistical Physics of Inference and Control Theory workshop organized in Granada (Spain), September 12-16, 2012 about stochastic optimal control theory and its links to inference, statistical mechanics and the theory
of large deviations. Functional data analysis and operatorial statistics workshop entitled Object, functional and structured data: towards next generation kernel-based methods, organized jointly to ICML 2012, June 30 in Edinburgh (UK): community of functional data analysis and operatorial statistics and those of kernel-based methods. PASCAL2 Overview Challenge Programme See separate presentation PASCAL2 Overview Curriculum development programme report Colin de la Higuera and Jos Balczar PASCAL2 Overview TML 2012
Second Workshop on Teaching Machine Learning Held in Edinburgh, on June 30th 2012, during ICML 40+ participants Organisers: Kurt Driessens, Elisa Fromont Agreement that Machine Learning is becoming a core topic in many universities, specifically hard to teach, and therefore useful to run this type of event. http://swarmlab.unimaas.nl/TeachingML/ PASCAL2 Overview Ghana-PASCAL
cooperation Ghana follow-up jointly funded via CDP and External Visitors programmes One Ghana student visited twice JLB at BCN JLB visited Accra twice Result: articles submitted PASCAL2 fifth Overview Year Review Ghana: lessons learnt
Workshops, bootcamp, etc were a success Large participation Impact (local) (press coverage, institutions) Key difficulties identified Local motivation is essential: the need must be felt: research at international level- is a very demanding activity Local support is essential: local government (academia, business) have to support specifically with long term commitment PASCAL2 Overview
Internal Visiting Claudio Gentile PASCAL2 Overview Internal Visiting Support visits between sites by Pascal2 members (travel and subsistence, no salary) Reshaped goal for final year of Pascal2: visit to prepare ground for longer term collaborations Final call to visit to industry scheme issued on July 2012 Activity in Y5: - 13 standard visits - 3 visits to industry (Telefonica, MSR Asia, Xerox) PASCAL2 Overview Videolectures Knowledge 4 All Foundation See separate presentation PASCAL2 Overview Legacy
PASCAL2 Overview Legacy projects Translectures: out of videolectures/K4All ComPLACS: pump-priming Excitement: RTE challenge PASCAL2 Overview
PASCAL Legacy Themes are emerging that could form the basis for a PASCAL3: Scaling to large data through linear or sublinear time learning: eg integrating with information retrieval to select relevant data Distributed learning: loosely connected suboptimisations converging to global optimum Intelligent search of large spaces as driver to attack big AI problems (cf MoGo) EU Watson, but general purpose and rapidly
reconfigurable Privacy and confidentiality Engaging the public in science PASCAL2 Overview PASCAL Legacy Breakdown of legacy: Videolectures, eprints, CDP: NCAST box significantly reduces costs: potential for charity funding from benefactor but also K4All (and JSI) involvement in EU
projects and UNESCO Workshops: possibility of co-funding with industry (eg workshop with Causata Ltd last September and planned for this year again), also co-funding with other projects Challenges: co-funding with sister projects or industry, eg CompLACS, EUCOGIII? Harvest/pump-priming: possible industry sponsorship but again may create difficulties Visits, Site funding, ...: very difficult to get support for this activity PASCAL2 Overview Some possible ideas ... EU funding for a further Network of Excellence
Would certainly need network to refocus and adjust membership Could aligned with several of the work programmes It seems Horizon 2020 will require significant SME involvement, but may be possibe PASCAL2 Overview Some possible ideas Company sponsorship: Give company access to brand: Aaa PASCAL3 Ask them to give base support as a charity donation through K4All
Enable calls for specific focussed research projects in areas of interest to them through Harvest/Pump-priming programmes Charity donations via K4All Find a donor prepared to support the network to develop such activities as videolectures and outreach to third world, while maintaining the research infrastructure PASCAL2 Overview
Machtelt has advised public authorities in Europe, Turkey and China on design of MRVA and legal frameworks; and provided training to industry, regulators, verifiers and accreditation bodies. 9-4-2019. Guide on designing accreditation & verification systems - introductory webinar.
Budgets, budgeting and budgetary control. A budget is an estimate or plan, expressed in quantitative terms, that outlines the consequences of the acquisition or use of an organization's resources. ... Communicating the details of the budget policy.
Overview of ProgramsDivision of Human Resource Development (HRD)NSF Education & Human Resources Directorate (NSF/EHR) "To increase the participation and advancement of underrepresented minorities and minority-serving institutions, women and girls, and persons with disabilities at every level of the science and...
The Higher School Certificate. is the culmination of a student's school career . is the highest educational award that can be achieved at secondary school in New South Wales. ... NSW HSC Testamur. Year 10 Subject Selection for Stage 6....
Conch Shell = New democracy on the island. Snake ... Lord of the Flies = A pig's head on a stick that becomes the physical acceptance of evil on the island. TERMS to REMEMBER ... paradise like, a setting that...
Ready to download the document? Go ahead and hit continue!