Auditing Automated Valuation Model Case: multivariate regression model

Auditing Automated Valuation Model Case: multivariate regression model

Auditing Automated Valuation Model Case: multivariate regression model for residential property Auditing automated valuation model case: multivariate regression model for residential property 1 Authors Auditing Automated Valuation Model Jamie Donovan Seppo Koponen Realia Management Oy Vice President MRICS,HypZert(MLV) AKA,KHK,LKV Realia Management Oy Analyst Valuation Advisory Authors Mikael Postila Kauko Viitanen Orava Funds Plc Head of Research, Investment

Analysis & Tenant Services Aalto University Professor, Head of the Department, Real Estate Economics and Valuation, Department of Real Estate, Planning and Geoinformatics FRICS Auditing automated valuation model case: multivariate regression model for residential property 2 Realia Management Oy A part of Realia Group Auditing automated valuation model case: multivariate regression model for residential property 3 Realia Management Oy A part of Realia Group Valuation Advisory Valuation services, Tailored advisory and consulting support services, Market research services

Realia Group is Nordics largest management and consulting group specialized in residential and commercial property transactions and in managing residential and commercial assets Auditing automated valuation model case: multivariate regression model for residential property 4 The case study Explorative case study Introspective We emphasise process description, exploring ways to improve and pinpoint issues we have encountered. Academic Positive/Descriptive Objective Subjective Normative Practice Auditing automated valuation model case: multivariate regression model for residential property We are not in a

position to give an objective, qualitative assesment of the success of the case 5 Description of Orava Residential REIT in-house AVM Purpose Monthly valuation of portfolio for board meetings Valuation for quarterly interim and end of year financial statement Valuation of proposed investments Reasoning behind in-house model Regulation Timeliness: all property assets can be valued at any given moment, without waiting external valuation. Company board should be aware of

developments at least monthly. Cost effectiness: assume a REIT has 100 properties, which are valued monthly update of valuation 1000 12*100*1000 = 1,2M !!!. In-house valuation team of 1-2 people is sufficiently large for this. AVM objectivity According to IFRS a company must value its assets at fair value. According to rather new IFRS standards company management must assess fair value i.e. mgmt must make its own assessment even though an external valuer (IVS, RICS, ) provides some calculation. Finnish law does not require external valuation Verification Finnish FSA and CPA (at the moment PwC) have Values are compared semi-annually against approved approach desktop valuation of chartered valuers (AKA/KHK) Valuation methodology is audited annually by external chartered valuer Auditing automated valuation model case: multivariate regression model for residential property 6 Description of Orava Residential REIT in-house AVM Methodology

Multivariate hedonic regression / OLS Specification OLS chosen since it 1) can be easily explained to Model specification changed only if a significant investors and board members, 2) is simple, no estimation/data issue is found need for high level econometric expertise, and 3) Changes are tested for the whole portfolio before can be verified and possible mistakes in applying estimation / data validation process are easy to find Same universal specification is used for every Problems: data related issues are difficult to property eliminate using regular OLS (better, but more Specification should be 1) theoretically sound, 2) difficult methods include: GLS, Kalman filter, GWR, fit data (yield reasonable valuations compared to neural networks, ) external valuation) and 3) minimise s.e. of Separate model done for every property regression Estimation and value calculation duplicated (->

spotting human errors) Data Asking prices (www.oikotie.fi portal); correction term calculated quarterly using Statistics Finland transaction price statistics Auditing automated valuation model case: multivariate regression model for residential property 7 ? What is an audit? Is this an audit? A second opinion. However, audit checks against audit criteria a more objective and systematic approach to assurance. systematic, independent and documented process for obtaining audit evidence (3.3) and evaluating it objectively to determine the extent to which the audit criteria (3.2) are fulfilled (ISO 19011:2011). According to ISA 500 and ISA 620, the AVM auditor can function as the reporting entitys expert or the primary (financial statement) auditors expert. (International Standards on Auditing) AVM valuations

AVM Auditor Audits of Financial Statement s Auditing automated valuation model case: multivariate regression model for residential property Public 8 Piecing standards and guidelines together IAAO ISO (International Organization for Standardization) IVS (International Valuation Standards) IAASB

RICS IFRS (International Auditing and Assurance Standards Board) (International Association of Assessing Officers) Auditing framewor k (draft) (International Financial Reporting Standards) AVM standard s (IVS) Publication s

A good overview of applicable standards Auditing automated valuation model case: multivariate regression model for residential property 9 Piecing standards and guidelines together Access to source material Knowledge of regression methodology Established Audit Criteria Auditing is made possible Valuation and local market Expertise Auditing automated valuation model case: multivariate regression model for residential property

1 0 Audit process of the Orava AVM The tenets of an audit Largely a question of pass or fail Towards standardisation You know what you get Auditing automated valuation model case: multivariate regression model for residential property 1 1 Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the

Audit Report and Statement Self-assesment of audit quality Beginning work Competency Other qualifications (independency) Documentation Audit plan Audit criteria Auditing automated valuation model case: multivariate regression model for residential property 1 2 Examples criteria used in Orava Audit For the whole portfolio, irrespective of the size of the portfolio, the sum of individual asset values must be within 5 % of the sum of asset values as valued by an independent valuer. For a sub-portfolio, the sum of values must be within 7.5 % of the sum of values as valued by an independent valuer. Single property assets (combination of multiple apartments at the same

address) must be valued within 15 % of the equivalent valuation by an independent valuer. Of the entire set of property assets, at least 80 % must pass this criteria. Auditing automated valuation model case: multivariate regression model for residential property 1 3 Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality Beginning work

Competency Documentation Audit plan Audit criteria Auditing automated valuation model case: multivariate regression model for residential property 1 4 Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality

The audit report structure Object and Contents of External Audit Regression Models in Property Valuation Automated Valuation Orava Automated Valuation Process (audit) Auditing Results and Audit Statement Auditing automated valuation model case: multivariate regression model for residential property 1 5 Scope of External Audit used in Orava Audit The purpose of the audit is to ascertain independently the balanced and quality and the true and fair treatment of the data and the results in respect to all parties involved. The object of audit is the automated property valuation model by Orava Fund The purpose of the automated valuation model is to define a market value for the properties owned by Orava Fund. The audit of the model is thus limited to the use of the model for the defining of fair value (market value) for the afore mentioned portfolio at the time of audit. The audit is based on the data and information obtained from Orava and other sources, in part verified against each other.

Auditing automated valuation model case: multivariate regression model for residential property 1 6 Scope of External Audit used in Orava Audit Definition of Audit The purpose of the audit is to ascertain independently the balanced and quality and the true and fair treatment of the data and the results in respect to all parties involved. Object of Audit The object of audit is the automated property valuation model by Orava Purpose and Scope Fund The purpose of the automated valuation model is to define a market value for the properties owned by Orava Fund. The audit of the model is thus limited to the use of the model for the defining of fair value (market value) for the afore mentioned portfolio at the time of audit. Sources The audit is based on the data and information obtained from Orava and other sources, in part verified against each other.

Auditing automated valuation model case: multivariate regression model for residential property 1 7 Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality The audit report structure

Object and Contents of External Audit Regression Models in Property Valuation Automated Valuation Orava Automated Valuation Process (audit) Auditing Results and Audit Statement Auditing automated valuation model case: multivariate regression model for residential property 1 8 Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality

Observing the AVM creation/implementation process Data collection and pre-processing Modelling Modelling post-processing and valuation AVM reporting Auditing automated valuation model case: multivariate regression model for residential property 1 9 Example table used in Orava Audit Auditing automated valuation model case: multivariate regression model for residential property 2 0 Audit process of the Orava AVM Beginning work Drafting report structure

Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality Observing the AVM creation/implementation process Data collection and pre-processing Modelling Modelling post-processing and valuation AVM reporting Auditing automated valuation model case: multivariate regression model for residential property 2 1 Audit process of the Orava AVM Beginning work

Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality Considering implications Regression analysis is prone to its own problems as do all methodologies for automatic price estimation Full evaluation requires understanding of the methods used in AVM audit should not be used merely as a stamp? Market/Fair Value definition is not problematic, however the use of AVMs for price indices is a more touchy subject. Assumptions! These are some of the considerations faced when doing an (AVM) audit. Auditing automated valuation model case: multivariate regression model for residential property

2 2 Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality Preparing the Audit Report and Statement Limitation of auditor liability Short form and long form Consider the audience! Auditing automated valuation model case: multivariate regression model for residential property 2 3

Audit process of the Orava AVM Beginning work Drafting report structure Considering implications Observing the AVM process Preparing the Audit Report and Statement Self-assesment of audit quality Self-assesment of audit quality ISO-standards 19011 IAASB handbook vol. 1, section for International standards on quality control IAASB publication Framework for audit Quality , Feb 2014. Auditing automated valuation model case: multivariate regression model for residential property 2 4

Most notable references ISO 19011:2011. (2011) Guidelines for auditing management systems. The International Organization for Standardization. IVS 2013. (2013) International Valuation Standards. International Valuation Standards Council. IVSC 2013. (2013) A Guide to the Audit Process for Professional Valuers. International Valuation Standards Council. IAASB. (2014/02) A framework for audit quality key elements that create an environment for audit quality. International Auditing and Assurance Standards Board Auditing automated valuation model case: multivariate regression model for residential property 2 Orava REIT AVM Audit 2013

AuditOfValuationModel2013.pdf Auditing automated valuation model case: multivariate regression model for residential property 2 Contact All input welcome! Jamie Donovan Helsinki, Finland Phone: +358 40 718 9135 [email protected] Auditing automated valuation model case: multivariate regression model for residential property 27 Some important terminology

Audit systematic, independent and documented process for obtaining audit evidence (3.3) and evaluating it objectively to determine the extent to which the audit criteria (3.2) are fulfilled (ISO 19011:2011). Internal/external audit Compliance Assurance Unqualified / qualified statement High / low confidence of Assurance Auditing criteria set of policies, procedures or requirements used as a reference against which audit evidence is compared (ISO 19011:2011). Auditing automated valuation model case: multivariate regression model for residential property 2 Variables of Orava Residential REIT in-house AVM

Size: 3rd degree polynomial Age: 3rd degree polynomial Decor/apt condition: 4 classes very good, good, fair, poor (dummy) Apartment sauna / proxy for equipment (dummy) Quarterly time dummies (8 quarters) Lot ownership dummy / min. 20 observations for leasehold in data Location level 1: zip code dummies Location level 2: 1sqkm or 4sqkm plane around the property

Plane can tilt both south-north & east-west 4sqkm plane used if obs individual properties < 15 in 1sqkm plane When there are known transaction prices for valued property they are assigned a property dummy. These are converted into asking prices using negotiation coefficient. Apartment specific dummies may be used when verified apartment specific asking prices exist. Rental house factor is assigned to all properties for which ownership exceeds 50% and construction year is before 1990 with no transactions. rental house dummy is calculated using average of property dummies (above) from other models Negotiation coefficient (currently 6,21% for cities with population < 100 000 and 4,94% above 100

000.) Auditing automated valuation model case: multivariate regression model for residential property 29

Recently Viewed Presentations

  • Introduction to WSU Accounting Presented by Tami Bidle

    Introduction to WSU Accounting Presented by Tami Bidle

    Why do we have all these rules? These laws and regulations provide the guidance framework for how the State does business. When properly implemented and followed, they will safeguard our resources, protect our staff, and provide internal controls to ensure...
  • Quick Start Guide :: vPC

    Quick Start Guide :: vPC

    This presentation will provide end-to-end configurations mapped directly to commonly deployed data center architecture topologies. In this cookbook style; quick start guide; configurations are broken down in an animated step by step process to a complete end-to-end good clean configuration...
  • Grant Preparation &amp; Submission at Mississippi State University

    Grant Preparation & Submission at Mississippi State University

    Cayusse424 - new! (1/11/14, 2-3 pm, Giles Computer Lab, Library) Agencies have 'pick up' sites. eRA Commons ID from SPA for NIH. NSPIRES ID for NASA *NSF FastLane. Portal for NSF. Need FastLane ID from SPA. Grant Preparation Guide (new...
  • Metric Conversions, Scientific Notation, and Dimensional Analysis

    Metric Conversions, Scientific Notation, and Dimensional Analysis

    Metric Conversions, Scientific Notation, and Dimensional Analysis International System of Units Built on a set of seven metric units, called base units (base units contain no prefix…examples are grams, meter, and liter) Prefixes are added to the names of SI...
  • Recounts tell someone what has happened. I use

    Recounts tell someone what has happened. I use

    We decided to try chocolate and margarine. Sample Slide Random Slides From This PowerPoint Show Sometimes, when I'm looking at changes, I keep a diary of the changes that happen. That's a recount too. Sample Slide Random Slides From This...
  • Tar Sands Are Bad! - University of Idaho

    Tar Sands Are Bad! - University of Idaho

    More Global Environment. Tailing Ponds are contaminating . the environment. (H2Oil) Immense amounts of water are used for tar sands operations - currently 349 million cubic meters per year, twice the amount of water used by the city of Calgary...
  • William Shakespeare

    William Shakespeare

    First, what do you know about William Shakespeare? What did he do? During which time period did he live? List anything that comes to mind! * William Shakespeare Shakespeare's Life Born April 23, 1564 in Stratford England He died on...
  • Assignment 4 My Final Concept Map Course: EDID

    Assignment 4 My Final Concept Map Course: EDID

    This program required me to do a lot of extra reading in order to complete assignments. The additional readings from books written by Marcy P. Driscoll, Smith & Ragan and Reiguluth and journal articles by Gagne, Merill and many others...