Approximate Message Passing

Size: px
Start display at page:

Download "Approximate Message Passing"

Transcription

1 Approximate Message Passing Mohsen Bayati, David Donoho, Adel Javanmard Iain Johnstone, Marc Lelarge, Arian Maleki, Andrea Montanari Stanford University December 8, 2012 Andrea Montanari (Stanford) AMP December 8, / 68

2 Statistical estimation y = f (; noise)! Unknown object y! Observations f ( ; noise)! Parametric model Problem: Estimate from observations y. Andrea Montanari (Stanford) AMP December 8, / 68

3 Statistical estimation y = f (; noise)! Unknown object y! Observations f ( ; noise)! Parametric model Problem: Estimate from observations y. Andrea Montanari (Stanford) AMP December 8, / 68

4 A broad convergence Statistics [Genomics,... ] Data mining [Collaborative ltering, Predictive analytics,... ] Signal processing [Compressive sampling,... ] Inverse problems [Medical imaging, Seismographic imaging,... ] +Data, + Computation, Exploit hidden structure Andrea Montanari (Stanford) AMP December 8, / 68

5 How should we think about these problems? Andrea Montanari (Stanford) AMP December 8, / 68

6 How should we think about these problems? Optimization? maximize Likelihood(jy) Complexity() `Separation principle' Modeler/statistician proposes convex cost function. Optimization expert proposes simple iterative algorithm. Run for 20 iterations and hope for the best. Andrea Montanari (Stanford) AMP December 8, / 68

7 How should we think about these problems? Beyond separation? y! ^ 1! ^ 2! ^ 3! : : : Constrained complexity per iteration Fixed number of iterations (say 20) What is minimum MSE achievable? Andrea Montanari (Stanford) AMP December 8, / 68

8 Outline A long example (algorithm + heuristics) A list of theorems/pointers Andrea Montanari (Stanford) AMP December 8, / 68

9 A long example Andrea Montanari (Stanford) AMP December 8, / 68

10 What type of example? Image processing (because they make nice gures) Compressed sensing (simple) WILL NOT MENTION SPARSITY! Andrea Montanari (Stanford) AMP December 8, / 68

11 What type of example? Image processing (because they make nice gures) Compressed sensing (simple) WILL NOT MENTION SPARSITY! Andrea Montanari (Stanford) AMP December 8, / 68

12 What type of example? Image processing (because they make nice gures) Compressed sensing (simple) WILL NOT MENTION SPARSITY! Andrea Montanari (Stanford) AMP December 8, / 68

13 An image = 2 C n Unknown object (n = : ) Andrea Montanari (Stanford) AMP December 8, / 68

14 Noiseless linear measurements y = A = A Want to reconstruct Andrea Montanari (Stanford) AMP December 8, / 68

15 Noiseless linear measurements y = A = A Want to reconstruct Andrea Montanari (Stanford) AMP December 8, / 68

16 Measurement structure A = e F R ef = subsampled Fourier matrix R = = random modulation! y 2 C m, m = 0:17 n Andrea Montanari (Stanford) AMP December 8, / 68

17 An approach popular in this community y = A + z ; z N(0; 2 I m m ) n 1 o p jy (jy) / exp ky 2 2 Ak2 2 p () my n 1 / exp y a A T 2 o ny a a=1 2 2 i=1 p i (i ) Andrea Montanari (Stanford) AMP December 8, / 68

18 An approach popular in this community y = A + z ; z N(0; 2 I m m ) n 1 o p jy (jy) / exp ky 2 2 Ak2 2 p () my n 1 / exp y a A T 2 o ny a a=1 2 2 i=1 p i (i ) Andrea Montanari (Stanford) AMP December 8, / 68

19 An approach popular in this community y = A + z ; z N(0; 2 I m m ) n 1 o p jy (jy) / exp ky 2 2 Ak2 2 p () my n 1 / exp y a A T 2 o ny a a=1 2 2 i=1 p i (i ) Andrea Montanari (Stanford) AMP December 8, / 68

20 Factor graph! 1 a m 1 i n p jy (jy) / my a=1 n 1 exp 2 2 y a A T 2 o ny a p i (i ) i=1 Use BP! Andrea Montanari (Stanford) AMP December 8, / 68

21 Many issues Anyone knows the prior distribution of natural images? Computation per iteration, memory (mn). Very loopy graph.... Let us try something simpler! Andrea Montanari (Stanford) AMP December 8, / 68

22 Many issues Anyone knows the prior distribution of natural images? Computation per iteration, memory (mn). Very loopy graph.... Let us try something simpler! Andrea Montanari (Stanford) AMP December 8, / 68

23 Constructing a rst estimate y = A Matched lter ^ 1 = 1 m Ay y Andrea Montanari (Stanford) AMP December 8, / 68

24 How good is this? E ^ 1 = (one line calculation) = Andrea Montanari (Stanford) AMP December 8, / 68

25 Check it out ^ 1 = A y y = = Does not look that good! Andrea Montanari (Stanford) AMP December 8, / 68

26 Check it out ^ 1 = A y y = = Does not look that good! Andrea Montanari (Stanford) AMP December 8, / 68

27 Idea = + `noise' Andrea Montanari (Stanford) AMP December 8, / 68

28 Idea = + Andrea Montanari (Stanford) AMP December 8, / 68

29 How big is the `noise'? Efk^ 1 k 2 2g = (two lines calculation) = 1 kk 2 2 Andrea Montanari (Stanford) AMP December 8, / 68

30 Matched lter blows up noise MSE 1 out = MSE in Andrea Montanari (Stanford) AMP December 8, / 68

31 Let's check MSE out MSE in Andrea Montanari (Stanford) AMP December 8, / 68

32 Denoising ^ 1 + z ; z i N(0; 1) Idea: Treat ^ 1 as eective observations in denoising Andrea Montanari (Stanford) AMP December 8, / 68

33 Denoising ^ 1 + z ; z i N(0; 1) Idea: Treat ^ 1 as eective observations in denoising Andrea Montanari (Stanford) AMP December 8, / 68

34 Denoising by nonlocal means by = + z ; ^i = P j W (i ; j )by j P j W (i ; j ) ; ( 1 if kpatch(i ; by) Patch(j ; by)k ; W (i ; j ) = 0 otherwise [Buades, Coll, Morel, 2005] ^ (by) Andrea Montanari (Stanford) AMP December 8, / 68

35 Denoising by nonlocal means by = + z ; ^i = P j W (i ; j )by j P j W (i ; j ) ; ( 1 if kpatch(i ; by) Patch(j ; by)k ; W (i ; j ) = 0 otherwise [Buades, Coll, Morel, 2005] ^ (by) Andrea Montanari (Stanford) AMP December 8, / 68

36 Denoising by nonlocal means by = + z ; ^i = P j W (i ; j )by j P j W (i ; j ) ; ( 1 if kpatch(i ; by) Patch(j ; by)k ; W (i ; j ) = 0 otherwise [Buades, Coll, Morel, 2005] ^ (by) Andrea Montanari (Stanford) AMP December 8, / 68

37 Patches i j Andrea Montanari (Stanford) AMP December 8, / 68

38 Will it work? ^ 2 = (^ 1 ) = (A y y) = Andrea Montanari (Stanford) AMP December 8, / 68

39 Let's try ^ 1 = A y y = ^2 = (A y y) = Better than garbage! Andrea Montanari (Stanford) AMP December 8, / 68

40 How much better? MSE out MSE in Andrea Montanari (Stanford) AMP December 8, / 68

41 How much better? ? MSE out c 2 p x c 1 x MSE in Andrea Montanari (Stanford) AMP December 8, / 68

42 Let us repeat the denoising experiment Andrea Montanari (Stanford) AMP December 8, / 68

43 Let us repeat the denoising experiment: y = + z y (y) = 1 = 0:5 = 0:25 = 0:12 Andrea Montanari (Stanford) AMP December 8, / 68

44 Quantitatively MSE out c 2 p x c 1 x MSE in Andrea Montanari (Stanford) AMP December 8, / 68

45 Quantitatively 0.1 MSE out c 2 p x c 1 x e MSE in Andrea Montanari (Stanford) AMP December 8, / 68

46 Approximate denoiser characterization MSE out = c p MSE in (enough for our purposes) (see also Maleki, Baraniuk, Narayan, 2012) (Arias-Castro, Willett, 2012) Andrea Montanari (Stanford) AMP December 8, / 68

47 What we achieved so far MSE out MSE in 0.1 MSE out MSE in Matched lter Denoiser Andrea Montanari (Stanford) AMP December 8, / 68

48 What we achieved so far MSE in MSE out 0.1 MSE out MSE in Matched lter Denoiser Andrea Montanari (Stanford) AMP December 8, / 68

49 What we achieved so far MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

50 What we achieved so far MSE new MSE old What about iterating? Andrea Montanari (Stanford) AMP December 8, / 68

51 What we achieved so far MSE new MSE old What about iterating? Andrea Montanari (Stanford) AMP December 8, / 68

52 How do we iterate? Will tell you later! Andrea Montanari (Stanford) AMP December 8, / 68

53 t = 1 ^ 1 = Andrea Montanari (Stanford) AMP December 8, / 68

54 t = 2 ^ 2 = Andrea Montanari (Stanford) AMP December 8, / 68

55 t = 3 ^ 3 = Andrea Montanari (Stanford) AMP December 8, / 68

56 t = MSE new Non obvious! MSE old Andrea Montanari (Stanford) AMP December 8, / 68

57 t = 4 ^ 4 = Andrea Montanari (Stanford) AMP December 8, / 68

58 t = MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

59 t = 5 ^ 5 = Andrea Montanari (Stanford) AMP December 8, / 68

60 t = MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

61 t = 6 ^ 6 = Andrea Montanari (Stanford) AMP December 8, / 68

62 t = MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

63 t = 7 ^ 7 = Andrea Montanari (Stanford) AMP December 8, / 68

64 t = MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

65 t = 8 ^ 8 = Andrea Montanari (Stanford) AMP December 8, / 68

66 t = MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

67 t = 9 ^ 9 = Andrea Montanari (Stanford) AMP December 8, / 68

68 t = MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

69 t = 0; 1; 2; 3; : : : ; 20 Andrea Montanari (Stanford) AMP December 8, / 68

70 t = 0; 1; 2; 3; : : : ; MSE new MSE old Andrea Montanari (Stanford) AMP December 8, / 68

71 How do we iterate? Andrea Montanari (Stanford) AMP December 8, / 68

72 Approximate Message Passing (AMP) ^ 2t = (^ 2t 1 ) ^ 2t+1 = ^ 2t + A y r t r t = y A^ 2t + b t r t 1 b t = 1 m div(^ 2t 1 ) (can be computed explicitly) [Thouless, Anderson, Palmer, 1977, Kabashima, 2003, Donoho, Maleki, Montanari, 2009, Donoho, Johnstone, Montanari, 2012] Andrea Montanari (Stanford) AMP December 8, / 68

73 Connection with Belief Propagation m i!j = m i + "i!j Linearize in "i!j Very dierent from naive mean eld! Andrea Montanari (Stanford) AMP December 8, / 68

74 Connection with Belief Propagation m i!j = m i + "i!j Linearize in "i!j Very dierent from naive mean eld! Andrea Montanari (Stanford) AMP December 8, / 68

75 Connection with Perturbation, Optimization, Statistics? Robustness wrt p X 2DistributionClass ( = minimax denoider in DistributionClass) Can rigorously track evolution over A random (What about A = A det + A rand?) Andrea Montanari (Stanford) AMP December 8, / 68

76 Connection with Perturbation, Optimization, Statistics? Robustness wrt p X 2DistributionClass ( = minimax denoider in DistributionClass) Can rigorously track evolution over A random (What about A = A det + A rand?) Andrea Montanari (Stanford) AMP December 8, / 68

77 A list of theorems/pointers Andrea Montanari (Stanford) AMP December 8, / 68

78 A list of theorems/pointers Connection with optimization [Bayati, Montanari, 2011] Minimax theory for sparse/block-sparse/tv [Donoho, Johnstone, Montanari, 2012] Bayesian reconstruction up to information dimension [Donoho, Javanmard, Montanari, 2012] Universality [Bayati, Lelarge, Montanari, 2012] Analysis of Generalized AMP [Javanmard, Montanari, 2012] Application to sparse PCA [In preparation, 2013] Andrea Montanari (Stanford) AMP December 8, / 68

79 Related work (Non-rigorous) replica method. [Tanaka 2002, Guo, Verdú 2005, Kabashima, Tanaka 2009, Rangan, Fletcher, Goyal 2009, Caire, Tulino, Shamai, Verdú ] Alternative argument for robust regression (e.g. min ky Ak 1 ) [Bean, Bickel, El Karoui, Lim, Yu 2012] Generalized linear models [Rangan 2011] Graphical model priors [Schniter et al ] Low-rank matrices [Rangan, Fletcher 2012] Andrea Montanari (Stanford) AMP December 8, / 68

80 Conclusion Andrea Montanari (Stanford) AMP December 8, / 68

81 Conclusion Can do message passing/bp without Bayesian assumptions! There is something between naive mean eld and BP Thanks! Andrea Montanari (Stanford) AMP December 8, / 68

82 Noise distribution? Frequency error Andrea Montanari (Stanford) AMP December 8, / 68

Online Algorithms and Competitive Analysis. Spring 2018

Online Algorithms and Competitive Analysis. Spring 2018 Online Algorithms and Competitive Analysis CS16: Introduction to Data Structures & Algorithms CS16: Introduction to Data Structures & Algorithms Spring 2018 Outline 1. Motivation 2. The Ski-Rental Problem

More information

Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015

Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015 Machine Learning Logistic Regression Hamid R. Rabiee Spring 2015 http://ce.sharif.edu/courses/93-94/2/ce717-1 / Agenda Probabilistic Classification Introduction to Logistic regression Binary logistic regression

More information

Intro. ANN & Fuzzy Systems. Lecture 36 GENETIC ALGORITHM (1)

Intro. ANN & Fuzzy Systems. Lecture 36 GENETIC ALGORITHM (1) Lecture 36 GENETIC ALGORITHM (1) Outline What is a Genetic Algorithm? An Example Components of a Genetic Algorithm Representation of gene Selection Criteria Reproduction Rules Cross-over Mutation Potential

More information

Network System Inference

Network System Inference Network System Inference Francis J. Doyle III University of California, Santa Barbara Douglas Lauffenburger Massachusetts Institute of Technology WTEC Systems Biology Final Workshop March 11, 2005 What

More information

Data Mining. Chapter 7: Score Functions for Data Mining Algorithms. Fall Ming Li

Data Mining. Chapter 7: Score Functions for Data Mining Algorithms. Fall Ming Li Data Mining Chapter 7: Score Functions for Data Mining Algorithms Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University The merit of score function Score function indicates

More information

CHAPTER 2 PROBLEM STATEMENT

CHAPTER 2 PROBLEM STATEMENT CHAPTER 2 PROBLEM STATEMENT Software metrics based heuristics support software quality engineering through improved scheduling and project control. It can be a key step towards steering the software testing

More information

The Combined Model of Gray Theory and Neural Network which is based Matlab Software for Forecasting of Oil Product Demand

The Combined Model of Gray Theory and Neural Network which is based Matlab Software for Forecasting of Oil Product Demand The Combined Model of Gray Theory and Neural Network which is based Matlab Software for Forecasting of Oil Product Demand Song Zhaozheng 1,Jiang Yanjun 2, Jiang Qingzhe 1 1State Key Laboratory of Heavy

More information

computational analysis of cell-to-cell heterogeneity in single-cell rna-sequencing data reveals hidden subpopulations of cells

computational analysis of cell-to-cell heterogeneity in single-cell rna-sequencing data reveals hidden subpopulations of cells computational analysis of cell-to-cell heterogeneity in single-cell rna-sequencing data reveals hidden subpopulations of cells Buettner et al., (2015) Nature Biotechnology, 1 32. doi:10.1038/nbt.3102 Saket

More information

Genomic models in bayz

Genomic models in bayz Genomic models in bayz Luc Janss, Dec 2010 In the new bayz version the genotype data is now restricted to be 2-allelic markers (SNPs), while the modeling option have been made more general. This implements

More information

1,3. not. mailed. mailed 1. mailed. not mailed. not mailed. mailed X 2 M X. p(s=subscribed) = 0.6 p(s=not subscribed) = 0.4

1,3. not. mailed. mailed 1. mailed. not mailed. not mailed. mailed X 2 M X. p(s=subscribed) = 0.6 p(s=not subscribed) = 0.4 A Decision Theoretic Approach to Targeted Advertising David axwell Chickering and David Heckerman icrosoft Research Redmond WA, 98052-6399 dmax@microsoft.com heckerma@microsoft.com Abstract A simple advertising

More information

Machine Learning in Computational Biology CSC 2431

Machine Learning in Computational Biology CSC 2431 Machine Learning in Computational Biology CSC 2431 Lecture 9: Combining biological datasets Instructor: Anna Goldenberg What kind of data integration is there? What kind of data integration is there? SNPs

More information

A Competitive Facility Location Game with Traffic Congestion Costs

A Competitive Facility Location Game with Traffic Congestion Costs A Competitive Facility Location Game with Traffic Congestion Costs Dr. Joseph Geunes & Dincer Konur University of Florida Industrial and Systems Engineering Department Outline Motivation and Introduction

More information

Reaction Paper Regarding the Flow of Influence and Social Meaning Across Social Media Networks

Reaction Paper Regarding the Flow of Influence and Social Meaning Across Social Media Networks Reaction Paper Regarding the Flow of Influence and Social Meaning Across Social Media Networks Mahalia Miller Daniel Wiesenthal October 6, 2010 1 Introduction One topic of current interest is how language

More information

Database Searching and BLAST Dannie Durand

Database Searching and BLAST Dannie Durand Computational Genomics and Molecular Biology, Fall 2013 1 Database Searching and BLAST Dannie Durand Tuesday, October 8th Review: Karlin-Altschul Statistics Recall that a Maximal Segment Pair (MSP) is

More information

DISTRIBUTED ARTIFICIAL INTELLIGENCE

DISTRIBUTED ARTIFICIAL INTELLIGENCE DISTRIBUTED ARTIFICIAL INTELLIGENCE LECTURE 3: PROBLEM MODELING INTRODUCTION There are different approaches for modeling of agent-based systems. The model that has gained most attention ti is modeling

More information

Why learn sequence database searching? Searching Molecular Databases with BLAST

Why learn sequence database searching? Searching Molecular Databases with BLAST Why learn sequence database searching? Searching Molecular Databases with BLAST What have I cloned? Is this really!my gene"? Basic Local Alignment Search Tool How BLAST works Interpreting search results

More information

TRANSPORTATION PROBLEM AND VARIANTS

TRANSPORTATION PROBLEM AND VARIANTS TRANSPORTATION PROBLEM AND VARIANTS Introduction to Lecture T: Welcome to the next exercise. I hope you enjoyed the previous exercise. S: Sure I did. It is good to learn new concepts. I am beginning to

More information

ENGG1811: Data Analysis using Excel 1

ENGG1811: Data Analysis using Excel 1 ENGG1811 Computing for Engineers Data Analysis using Excel (weeks 2 and 3) Data Analysis Histogram Descriptive Statistics Correlation Solving Equations Matrix Calculations Finding Optimum Solutions Financial

More information

Social Recommendation: A Review

Social Recommendation: A Review Noname manuscript No. (will be inserted by the editor) Social Recommendation: A Review Jiliang Tang Xia Hu Huan Liu Received: date / Accepted: date Abstract Recommender systems play an important role in

More information

Optimization of High-Frequency Trading Systems

Optimization of High-Frequency Trading Systems Optimization of High-Frequency Trading Systems David Sweet dsweet@andamooka.org - ex, trading strategy: buy/sell for profit - ex, execution system: fill an order for a customer High-Frequency Trading Dynamics/signals

More information

Machine learning in neuroscience

Machine learning in neuroscience Machine learning in neuroscience Bojan Mihaljevic, Luis Rodriguez-Lujan Computational Intelligence Group School of Computer Science, Technical University of Madrid 2015 IEEE Iberian Student Branch Congress

More information

Rank hotels on Expedia.com to maximize purchases

Rank hotels on Expedia.com to maximize purchases Rank hotels on Expedia.com to maximize purchases Nishith Khantal, Valentina Kroshilina, Deepak Maini December 14, 2013 1 Introduction For an online travel agency (OTA), matching users to hotel inventory

More information

VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY. A seminar report on GENETIC ALGORITHMS.

VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY. A seminar report on GENETIC ALGORITHMS. VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY A seminar report on GENETIC ALGORITHMS Submitted by Pranesh S S 2SD06CS061 8 th semester DEPARTMENT OF COMPUTER SCIENCE

More information

Ensemble Modeling. Toronto Data Mining Forum November 2017 Helen Ngo

Ensemble Modeling. Toronto Data Mining Forum November 2017 Helen Ngo Ensemble Modeling Toronto Data Mining Forum November 2017 Helen Ngo Agenda Introductions Why Ensemble Models? Simple & Complex ensembles Thoughts: Post-real-life Experimentation Downsides of Ensembles

More information

On Robust Monetary Policy with Structural Uncertainty

On Robust Monetary Policy with Structural Uncertainty Diebold, F.X. (2005), "On Robust Monetary Policy with Structural Uncertainty," in J. Faust, A. Orphanedes and D. Reifschneider (eds.), Models and Monetary Policy: Research in the Tradition of Dale Henderson,

More information

2 Maria Carolina Monard and Gustavo E. A. P. A. Batista

2 Maria Carolina Monard and Gustavo E. A. P. A. Batista Graphical Methods for Classifier Performance Evaluation Maria Carolina Monard and Gustavo E. A. P. A. Batista University of São Paulo USP Institute of Mathematics and Computer Science ICMC Department of

More information

Predicting Yelp Restaurant Reviews

Predicting Yelp Restaurant Reviews Predicting Yelp Restaurant Reviews Wael Farhan UCSD: A53070918 9500 Gilman Drive La Jolla, CA, 92093 wfarhan@eng.ucsd.edu ABSTRACT Starting a restaurant is a tricky business. Restaurant owners have to

More information

A Propagation-based Algorithm for Inferring Gene-Disease Associations

A Propagation-based Algorithm for Inferring Gene-Disease Associations A Propagation-based Algorithm for Inferring Gene-Disease Associations Oron Vanunu Roded Sharan Abstract: A fundamental challenge in human health is the identification of diseasecausing genes. Recently,

More information

Compressed Sensing in Information Processing

Compressed Sensing in Information Processing The first report colloquium on the priority programme Compressed Sensing in Information Processing RWTH Aachen University - ICT cubes, July 4-5, 2016 July 4th (14:00-16: 00) Low-Rank Matrix Recovery using

More information

DOES DATA MINING IMPROVE BUSINESS FORECASTING?

DOES DATA MINING IMPROVE BUSINESS FORECASTING? DOES DATA MINING IMPROVE BUSINESS FORECASTING? June 13, 1998 David Chereb, Ph.D. Prepared for: THE 18 TH INTERNATIONAL SYMPOSIUM ON FORECASTING Edinburgh, Scotland INTRODUCTION The purpose of this article

More information

Getting Started with OptQuest

Getting Started with OptQuest Getting Started with OptQuest What OptQuest does Futura Apartments model example Portfolio Allocation model example Defining decision variables in Crystal Ball Running OptQuest Specifying decision variable

More information

Active Learning for Logistic Regression

Active Learning for Logistic Regression Active Learning for Logistic Regression Andrew I. Schein The University of Pennsylvania Department of Computer and Information Science Philadelphia, PA 19104-6389 USA ais@cis.upenn.edu April 21, 2005 What

More information

Intro Logistic Regression Gradient Descent + SGD

Intro Logistic Regression Gradient Descent + SGD Case Study 1: Estimating Click Probabilities Intro Logistic Regression Gradient Descent + SGD Machine Learning for Big Data CSE547/STAT548, University of Washington Sham Kakade March 29, 2016 1 Ad Placement

More information

Advanced Introduction to Machine Learning

Advanced Introduction to Machine Learning Advanced Introduction to Machine Learning 10715, Fall 2014 Structured Sparsity, with application in Computational Genomics Eric Xing Lecture 3, September 15, 2014 Reading: Eric Xing @ CMU, 2014 1 Structured

More information

Data Mining and Applications in Genomics

Data Mining and Applications in Genomics Data Mining and Applications in Genomics Lecture Notes in Electrical Engineering Volume 25 For other titles published in this series, go to www.springer.com/series/7818 Sio-Iong Ao Data Mining and Applications

More information

Applying Regression Techniques For Predictive Analytics Paviya George Chemparathy

Applying Regression Techniques For Predictive Analytics Paviya George Chemparathy Applying Regression Techniques For Predictive Analytics Paviya George Chemparathy AGENDA 1. Introduction 2. Use Cases 3. Popular Algorithms 4. Typical Approach 5. Case Study 2016 SAPIENT GLOBAL MARKETS

More information

Modeling of competition in revenue management Petr Fiala 1

Modeling of competition in revenue management Petr Fiala 1 Modeling of competition in revenue management Petr Fiala 1 Abstract. Revenue management (RM) is the art and science of predicting consumer behavior and optimizing price and product availability to maximize

More information

Adaptive Mechanism Design: A Metalearning Approach

Adaptive Mechanism Design: A Metalearning Approach In Proceedings of the International Conference on Electronic Commerce (ICEC 06), Fredericton, Canada, August 2006. Adaptive Mechanism Design: A Metalearning Approach ABSTRACT David Pardoe Peter Stone Department

More information

Predicting Yelp Ratings From Business and User Characteristics

Predicting Yelp Ratings From Business and User Characteristics Predicting Yelp Ratings From Business and User Characteristics Jeff Han Justin Kuang Derek Lim Stanford University jeffhan@stanford.edu kuangj@stanford.edu limderek@stanford.edu I. Abstract With online

More information

Tutorial Segmentation and Classification

Tutorial Segmentation and Classification MARKETING ENGINEERING FOR EXCEL TUTORIAL VERSION v171025 Tutorial Segmentation and Classification Marketing Engineering for Excel is a Microsoft Excel add-in. The software runs from within Microsoft Excel

More information

Calculating a Consensus of Opinion

Calculating a Consensus of Opinion Calculating a Consensus of Opinion Best Practice Authored by: Aaron Ramert, STAT COE 30 August 2017 The goal of the STAT COE is to assist in developing rigorous, defensible test strategies to more effectively

More information

Price Prediction Evolution: from Economic Model to Machine Learning

Price Prediction Evolution: from Economic Model to Machine Learning Price Prediction Evolution: from Economic Model to Machine Learning By Hugh Ren (hughrxy), Xiangguang Zheng (zhengxg3), Erico Cruz Lemus (ecruzlem) Abstract This report is intended to illustrate timeline

More information

Time Series Motif Discovery

Time Series Motif Discovery Time Series Motif Discovery Bachelor s Thesis Exposé eingereicht von: Jonas Spenger Gutachter: Dr. rer. nat. Patrick Schäfer Gutachter: Prof. Dr. Ulf Leser eingereicht am: 10.09.2017 Contents 1 Introduction

More information

Quality Function Deployment

Quality Function Deployment Quality Function Deployment Customer Driven Product Development Illustrated by Example Define Identify CCRs Design Optimize for 6 Validate Define the Project Business Case, Opportunity Statement, Goal,

More information

Life Cycle Assessment A product-oriented method for sustainability analysis. UNEP LCA Training Kit Module f Interpretation 1

Life Cycle Assessment A product-oriented method for sustainability analysis. UNEP LCA Training Kit Module f Interpretation 1 Life Cycle Assessment A product-oriented method for sustainability analysis UNEP LCA Training Kit Module f Interpretation 1 ISO 14040 framework Life cycle assessment framework Goal and scope definition

More information

Two Lectures on Information Design

Two Lectures on Information Design Two Lectures on nformation Design 10th DSE Winter School Stephen Morris December 2015 nformation Design Usually in information economics, we ask what can happen for a xed information structure... nformation

More information

82 R. FISCHER AND L. MIRMAN sectors. It is then easy to make comparisons between the cooperative and the noncooperative solutions. This formulation al

82 R. FISCHER AND L. MIRMAN sectors. It is then easy to make comparisons between the cooperative and the noncooperative solutions. This formulation al c Journal of Applied Mathematics & Decision Sciences, 1(2), 81{87 (1997) Reprints Available directly from the Editor. Printed in New Zealand. Limit Policies in N-Sector Dynamic Growth Games with Externalities

More information

Resource Decisions in Software Development Using Risk Assessment Model

Resource Decisions in Software Development Using Risk Assessment Model Proceedings of the 39th Hawaii International Conference on System Sciences - 6 Resource Decisions in Software Development Using Risk Assessment Model Wiboon Jiamthubthugsin Department of Computer Engineering

More information

BIOINFORMATICS AND SYSTEM BIOLOGY (INTERNATIONAL PROGRAM)

BIOINFORMATICS AND SYSTEM BIOLOGY (INTERNATIONAL PROGRAM) BIOINFORMATICS AND SYSTEM BIOLOGY (INTERNATIONAL PROGRAM) PROGRAM TITLE DEGREE TITLE Master of Science Program in Bioinformatics and System Biology (International Program) Master of Science (Bioinformatics

More information

Kanban kick- start (v2)

Kanban kick- start (v2) Kanban kick- start (v2) By Tomas Björkholm at Crisp, October 2011 INTRODUCTION... 1 AN APPROACH TO GET STARTED WITH KANBAN... 2 STEP 1 GET TO KNOW YOUR SYSTEM... 2 STEP 2 IDENTIFY YOUR SOURCES AND PRIORITIZE...

More information

Tactical Planning using Heuristics

Tactical Planning using Heuristics Tactical Planning using Heuristics Roman van der Krogt a Leon Aronson a Nico Roos b Cees Witteveen a Jonne Zutt a a Delft University of Technology, Faculty of Information Technology and Systems, P.O. Box

More information

What is the Difference between a Human and a Chimp? T. M. Murali

What is the Difference between a Human and a Chimp? T. M. Murali What is the Difference between a Human and a Chimp? T. M. Murali murali@cs.vt.edu http://bioinformatics.cs.vt.edu/ murali Introduction to Computational Biology and Bioinformatics (CS 3824) March 25 and

More information

Scheduling of Genetic Analysis Workflows in Grid Environments

Scheduling of Genetic Analysis Workflows in Grid Environments Aalto University School of Science Degree programme in Engineering Physics and Mathematics Scheduling of Genetic Analysis Workflows in Grid Environments Bachelor's Thesis 29.4.2015 Arttu Voutilainen The

More information

Dynamic Programming Algorithms

Dynamic Programming Algorithms Dynamic Programming Algorithms Sequence alignments, scores, and significance Lucy Skrabanek ICB, WMC February 7, 212 Sequence alignment Compare two (or more) sequences to: Find regions of conservation

More information

What about streaming data?

What about streaming data? What about streaming data? 1 The Stream Model Data enters at a rapid rate from one or more input ports Such data are called stream tuples The system cannot store the entire (infinite) stream Distribution

More information

TIMETABLING EXPERIMENTS USING GENETIC ALGORITHMS. Liviu Lalescu, Costin Badica

TIMETABLING EXPERIMENTS USING GENETIC ALGORITHMS. Liviu Lalescu, Costin Badica TIMETABLING EXPERIMENTS USING GENETIC ALGORITHMS Liviu Lalescu, Costin Badica University of Craiova, Faculty of Control, Computers and Electronics Software Engineering Department, str.tehnicii, 5, Craiova,

More information

Data Mining Applications with R

Data Mining Applications with R Data Mining Applications with R Yanchang Zhao Senior Data Miner, RDataMining.com, Australia Associate Professor, Yonghua Cen Nanjing University of Science and Technology, China AMSTERDAM BOSTON HEIDELBERG

More information

Dealing with Uncertainty in Optimization Models using AIMMS

Dealing with Uncertainty in Optimization Models using AIMMS Dealing with Uncertainty in Optimization Models using AIMMS Dr. Ovidiu Listes Senior Consultant AIMMS Analytics and Optimization Outline Analytics, Optimization, Uncertainty Use Case: Power System Expansion

More information

Introduction to gene expression microarray data analysis

Introduction to gene expression microarray data analysis Introduction to gene expression microarray data analysis Outline Brief introduction: Technology and data. Statistical challenges in data analysis. Preprocessing data normalization and transformation. Useful

More information

Modeling and Predicting User Interests based on Taxonomy. Makoto Nakatsuji

Modeling and Predicting User Interests based on Taxonomy. Makoto Nakatsuji Modeling and Predicting User Interests based on Taxonomy Makoto Nakatsuji Abstract In the thesis, we analyze user interests based on a domain specific taxonomy. We propose modeling user interests and

More information

Bayesian Variable Selection and Data Integration for Biological Regulatory Networks

Bayesian Variable Selection and Data Integration for Biological Regulatory Networks Bayesian Variable Selection and Data Integration for Biological Regulatory Networks Shane T. Jensen Department of Statistics The Wharton School, University of Pennsylvania stjensen@wharton.upenn.edu Gary

More information

Winsor Approach in Regression Analysis. with Outlier

Winsor Approach in Regression Analysis. with Outlier Applied Mathematical Sciences, Vol. 11, 2017, no. 41, 2031-2046 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ams.2017.76214 Winsor Approach in Regression Analysis with Outlier Murih Pusparum Qasa

More information

Predictive Planning for Supply Chain Management

Predictive Planning for Supply Chain Management Predictive Planning for Supply Chain Management David Pardoe and Peter Stone Department of Computer Sciences The University of Texas at Austin {dpardoe, pstone}@cs.utexas.edu Abstract Supply chains are

More information

Wind turbine vibration study: a data driven methodology

Wind turbine vibration study: a data driven methodology University of Iowa Iowa Research Online Theses and Dissertations Fall 2009 Wind turbine vibration study: a data driven methodology Zijun Zhang University of Iowa Copyright 2009 Zijun Zhang This thesis

More information

Forecasting Introduction Version 1.7

Forecasting Introduction Version 1.7 Forecasting Introduction Version 1.7 Dr. Ron Tibben-Lembke Sept. 3, 2006 This introduction will cover basic forecasting methods, how to set the parameters of those methods, and how to measure forecast

More information

Improving the Accuracy of Base Calls and Error Predictions for GS 20 DNA Sequence Data

Improving the Accuracy of Base Calls and Error Predictions for GS 20 DNA Sequence Data Improving the Accuracy of Base Calls and Error Predictions for GS 20 DNA Sequence Data Justin S. Hogg Department of Computational Biology University of Pittsburgh Pittsburgh, PA 15213 jsh32@pitt.edu Abstract

More information

Bilateral and Multilateral Exchanges for Peer-Assisted Content Distribution

Bilateral and Multilateral Exchanges for Peer-Assisted Content Distribution 1290 IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 19, NO. 5, OCTOBER 2011 Bilateral and Multilateral Exchanges for Peer-Assisted Content Distribution Christina Aperjis, Ramesh Johari, Member, IEEE, and Michael

More information

New Year, New NIH Expectations: Are You Ready?

New Year, New NIH Expectations: Are You Ready? New Year, New NIH Expectations: Are You Ready? Jennifer A. Croker, PhD Admin. Director, CCTS Research Commons David T. Redden, PhD Co-Director, CCTS BERD Chair, Department of Biostatistics December 2,

More information

The feasibility of using field experiments to optimize marketing decisions remains relatively unstudied. We

The feasibility of using field experiments to optimize marketing decisions remains relatively unstudied. We MANAGEMENT SCIENCE Vol. 61, No. 7, July 215, pp. 1722 174 ISSN 25-199 (print) ó ISSN 1526-551 (online) http://dx.doi.org/1.1287/mnsc.214.266 215 INFORMS The Value of Field Experiments Jimmy Q. Li Laboratory

More information

Module 1 Introduction. IIT, Bombay

Module 1 Introduction. IIT, Bombay Module 1 Introduction Lecture 1 Need Identification and Problem Definition Instructional objectives The primary objective of this lecture module is to outline how to identify the need and define the problem

More information

Accurate Prediction for Atomic-Level Protein Design and its Application in Diversifying the Near-Optimal Sequence Space

Accurate Prediction for Atomic-Level Protein Design and its Application in Diversifying the Near-Optimal Sequence Space Page 1 of 51 Research Article Proteins: Structure, Function and Bioinformatics DOI 1.12/prot.2228 Accurate Prediction for Atomic-Level Protein Design and its Application in Diversifying the Near-Optimal

More information

Using Decision Tree to predict repeat customers

Using Decision Tree to predict repeat customers Using Decision Tree to predict repeat customers Jia En Nicholette Li Jing Rong Lim Abstract We focus on using feature engineering and decision trees to perform classification and feature selection on the

More information

MANY important decisions are based on measurement results. Some of. Measurement Uncertainty. Chapter 1

MANY important decisions are based on measurement results. Some of. Measurement Uncertainty. Chapter 1 1 Chapter 1 Measurement Uncertainty Decisions, Confidence and Measurement Uncertainty Comparability and Traceability ISO 17025 MANY important decisions are based on measurement results. Some of them are

More information

ECLT 5810 E-Commerce Data Mining Techniques - Introduction. Prof. Wai Lam

ECLT 5810 E-Commerce Data Mining Techniques - Introduction. Prof. Wai Lam ECLT 5810 E-Commerce Data Mining Techniques - Introduction Prof. Wai Lam Data Opportunities Business infrastructure have improved the ability to collect data Virtually every aspect of business is now open

More information

Churn Prediction Model Using Linear Discriminant Analysis (LDA)

Churn Prediction Model Using Linear Discriminant Analysis (LDA) IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 18, Issue 5, Ver. IV (Sep. - Oct. 2016), PP 86-93 www.iosrjournals.org Churn Prediction Model Using Linear Discriminant

More information

Genetic Algorithms in Matrix Representation and Its Application in Synthetic Data

Genetic Algorithms in Matrix Representation and Its Application in Synthetic Data Genetic Algorithms in Matrix Representation and Its Application in Synthetic Data Yingrui Chen *, Mark Elliot ** and Joe Sakshaug *** * ** University of Manchester, yingrui.chen@manchester.ac.uk University

More information

GLMs the Good, the Bad, and the Ugly Ratemaking and Product Management Seminar March Christopher Cooksey, FCAS, MAAA EagleEye Analytics

GLMs the Good, the Bad, and the Ugly Ratemaking and Product Management Seminar March Christopher Cooksey, FCAS, MAAA EagleEye Analytics Antitrust Notice The Casualty Actuarial Society is committed to adhering strictly to the letter and spirit of the antitrust laws. Seminars conducted under the auspices of the CAS are designed solely to

More information

Brian Macdonald Big Data & Analytics Specialist - Oracle

Brian Macdonald Big Data & Analytics Specialist - Oracle Brian Macdonald Big Data & Analytics Specialist - Oracle Improving Predictive Model Development Time with R and Oracle Big Data Discovery brian.macdonald@oracle.com Copyright 2015, Oracle and/or its affiliates.

More information

Data Mining. CS57300 Purdue University. March 27, 2018

Data Mining. CS57300 Purdue University. March 27, 2018 Data Mining CS57300 Purdue University March 27, 2018 1 Recap last class So far we have seen how to: Test a hypothesis in batches (A/B testing) Test multiple hypotheses (Paul the Octopus-style) 2 The New

More information

Notes on Introduction to Contract Theory

Notes on Introduction to Contract Theory Notes on Introduction to Contract Theory John Morgan Haas School of Business and Department of Economics University of California, Berkeley 1 Overview of the Course This is a readings course. The lectures

More information

Predicting the Odds of Getting Retweeted

Predicting the Odds of Getting Retweeted Predicting the Odds of Getting Retweeted Arun Mahendra Stanford University arunmahe@stanford.edu 1. Introduction Millions of people tweet every day about almost any topic imaginable, but only a small percent

More information

A selective look a modelling challenges in energy associated with large scale

A selective look a modelling challenges in energy associated with large scale A selective look a modelling challenges in energy associated with large scale Part A. 4 examples of large scale engineering/or models Part B. Reflections on Data Science in optimisation ATI Data Science

More information

Read Mapping and Variant Calling. Johannes Starlinger

Read Mapping and Variant Calling. Johannes Starlinger Read Mapping and Variant Calling Johannes Starlinger Application Scenario: Personalized Cancer Therapy Different mutations require different therapy Collins, Meredith A., and Marina Pasca di Magliano.

More information

Building Energy Modeling Using Artificial Neural Networks

Building Energy Modeling Using Artificial Neural Networks Energy Research Journal Original Research Paper Building Energy Modeling Using Artificial Neural Networks Maya Arida, Nabil Nassif, Rand Talib and Taher Abu-Lebdeh Department of Civil, Architectural and

More information

Results of the Enterprise Project. Jussi Stader AIAI-TR-209. September 1996

Results of the Enterprise Project. Jussi Stader AIAI-TR-209. September 1996 Results of the Enterprise Project Jussi Stader AIAI-TR-209 September 1996 This paper appears in the proceedings of Expert Systems '96, the 16th Annual Conference of the British Computer Society Specialist

More information

LOAD FORECASTING FOR POWER SYSTEM PLANNING AND OPERATION USING ARTIFICIAL NEURAL NETWORK AT AL BATINAH REGION OMAN

LOAD FORECASTING FOR POWER SYSTEM PLANNING AND OPERATION USING ARTIFICIAL NEURAL NETWORK AT AL BATINAH REGION OMAN Journal of Engineering Science and Technology Vol. 7, No. 4 (01) 498-504 School of Engineering, Taylor s University LOAD FORECASTING FOR POWER SYSTEM PLANNING AND OPERATION USING ARTIFICIAL NEURAL NETWORK

More information

Analytical Capability Security Compute Ease Data Scale Price Users Traditional Statistics vs. Machine Learning In-Memory vs. Shared Infrastructure CRAN vs. Parallelization Desktop vs. Remote Explicit vs.

More information

Capacity Dilemma: Economic Scale Size versus. Demand Fulfillment

Capacity Dilemma: Economic Scale Size versus. Demand Fulfillment Capacity Dilemma: Economic Scale Size versus Demand Fulfillment Chia-Yen Lee (cylee@mail.ncku.edu.tw) Institute of Manufacturing Information and Systems, National Cheng Kung University Abstract A firm

More information

Pricing with Market Power

Pricing with Market Power Chapter 7 Pricing with Market Power 7.1 Motives and objectives Broadly The model of perfect competition is extreme (and hence wonderfully powerful and simple) because of its assumption that each firm believes

More information

Journal of Global Research in Computer Science PREMATURE CONVERGENCE AND GENETIC ALGORITHM UNDER OPERATING SYSTEM PROCESS SCHEDULING PROBLEM

Journal of Global Research in Computer Science PREMATURE CONVERGENCE AND GENETIC ALGORITHM UNDER OPERATING SYSTEM PROCESS SCHEDULING PROBLEM Volume, No. 5, December 00 Journal of Global Research in Computer Science RESEARCH PAPER Available Online at www.jgrcs.info PREMATURE CONVERGENCE AND GENETIC ALGORITHM UNDER OPERATING SYSTEM PROCESS SCHEDULING

More information

On polyclonality of intestinal tumors

On polyclonality of intestinal tumors Michael A. University of Wisconsin Chaos and Complex Systems April 2006 Thanks Linda Clipson W.F. Dove Rich Halberg Stephen Stanhope Ruth Sullivan Andrew Thliveris Outline Bio Three statistical questions

More information

Using Weights in the Analysis of Survey Data

Using Weights in the Analysis of Survey Data Using Weights in the Analysis of Survey Data David R. Johnson Department of Sociology Population Research Institute The Pennsylvania State University November 2008 What is a Survey Weight? A value assigned

More information

New restaurants fail at a surprisingly

New restaurants fail at a surprisingly Predicting New Restaurant Success and Rating with Yelp Aileen Wang, William Zeng, Jessica Zhang Stanford University aileen15@stanford.edu, wizeng@stanford.edu, jzhang4@stanford.edu December 16, 2016 Abstract

More information

Generative Models for Networks and Applications to E-Commerce

Generative Models for Networks and Applications to E-Commerce Generative Models for Networks and Applications to E-Commerce Patrick J. Wolfe (with David C. Parkes and R. Kang-Xing Jin) Division of Engineering and Applied Sciences Department of Statistics Harvard

More information

Stochastic Convex Sparse Principal Component Analysis

Stochastic Convex Sparse Principal Component Analysis Stochastic Convex Sparse Principal Component Analysis Inci M. Baytas 1, Kaixiang Lin 1, Fei Wang 2, Anil K. Jain 1 and Jiayu Zhou 1 1 Michigan State University 2 Cornell University 1 Principal Component

More information

WE consider the general ranking problem, where a computer

WE consider the general ranking problem, where a computer 5140 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 11, NOVEMBER 2008 Statistical Analysis of Bayes Optimal Subset Ranking David Cossock and Tong Zhang Abstract The ranking problem has become increasingly

More information

Estimation, Forecasting and Overbooking

Estimation, Forecasting and Overbooking Estimation, Forecasting and Overbooking Block 2 Zaenal Akbar (zaenal.akbar@sti2.at) Copyright 2008 STI INNSBRUCK Agenda Topics: 1. Estimation and Forecasting 2. Overbooking Tuesday, 21.03.2017 1 146418

More information

Genetic Algorithm and Neural Network

Genetic Algorithm and Neural Network Proceedings of the 7th WSEAS International Conference on Applied Informatics and Communications, Athens, Greece, August 24-26, 2007 345 Genetic Algorithm and Neural Network JIRI STASTNY*, VLADISLAV SKORPIL**

More information

100 Training Error Testing Error second 1 Intersection 1 Turn. Distance Exchange Value in Feet 500. Percentage Correct

100 Training Error Testing Error second 1 Intersection 1 Turn. Distance Exchange Value in Feet 500. Percentage Correct Interactive Renement of Route Preferences for Driving Seth Rogers and Pat Langley (rogers, langley)@rtna.daimlerbenz.com Daimler-Benz Research and Technology Center 1510 Page Mill Road, Palo Alto, CA 94304-1135

More information

Informative SNP Selection and Validation

Informative SNP Selection and Validation Georgia State University ScholarWorks @ Georgia State University Computer Science Theses Department of Computer Science 8-3-2007 Informative SNP Selection and Validation Diana Mohan Babu Follow this and

More information