Examination Report Shares Findings That Will Drive the Future of the ARE

10 June 2013

Washington, DC—The Examination Report, the third in a special series of publications from the National Council of Architectural Registration Boards (NCARB), shares examination-related findings from the most comprehensive practice analysis ever undertaken for the architecture profession. Required by all 54 U.S. jurisdictions, the ARE plays a critical role in assessing the knowledge, skills, and abilities to provide the various services required for the independent practice of architecture.

“In addition to informing interim updates to the current version of the ARE, the Practice Analysis will serve as a foundation for the development of future versions of the exam,” said NCARB CEO Michael J. Armstrong. “The findings will also have a significant impact on our exploration of alternative pathways to licensure that further blend the three traditional components of education, internship, and examination.”

The Examination Report encompasses extensive data collected from three examination-specific surveys:

  • Architects were asked to indicate how frequently they performed the task in the past year and to rate the level of importance of the competent performance of the task by a recently licensed architect practicing independently;

  • In this survey, architects were presented with two similar but distinct questions. The first question is very common in practice analyses and asks when each knowledge/skill was acquired. The second question asked the same respondents to identify when each knowledge/skill should be acquired; and

  • In the third survey, architects were asked to rate the importance of each knowledge/skill to a recently licensed architect practicing independently and at what level they typically use the knowledge/skill when performing their job.
Highlights of Examination Findings

The Practice Analysis provided an opportunity to analyze the ARE in relation to the contemporary practice of architecture. The resulting findings were the product of five key areas of analysis:

  • Level of Importance – The survey indicates that 129 of the 132 knowledge/skills, and 106 of the 110 tasks, were rated as “important” or greater by architects who completed the survey. Three of these K/S and tasks were rated as “critically important” and are directly related to the protection of the public health, safety, and welfare.

  • Point of Knowledge/Skill Acquisition – When comparing level of importance with point of acquisition, 15 knowledge/skills were identified as “important” or greater and also identified as being acquired after licensure by more than 50 percent of architects completing the survey. These 15 knowledge/skills primarily deal with practice and project management issues and are vital to competent practice; therefore, their acquisition should be better supported during education and internship.

  • Level of Knowledge/Skill Use – Architects were asked to rate the level at which they use each knowledge/skill. “Apply” was the most frequently selected response at 42.5 percent. “Evaluate” and “Understand” were evenly split at 26.0 percent and 25.7 percent, respectively. Only 5.8 percent of architects indicated they did not use the knowledge/skill in their job. This data will be used to support item writers in the creation of more relevant items/questions for the examination.

  • Frequency of Task Performance – Over 70 percent of the tasks included in the survey were indicated by architects as being performed in the past year. Most tasks were rated as being performed “quarterly” (20.4 percent) or “monthly” (19.0 percent). This data will be used to refine the content and distribution of items included in the ARE. Ten tasks rated “important” or greater were identified as “not performed” in the past year by more than 50 percent of architects. Additional analysis by various NCARB committees is warranted to better understand the nature of those tasks.

  • Sub-group Analysis – Response rates based on years of licensed experience and size of firm revealed little variation. More experienced practitioners tended to report a slightly higher level of ability than those recently licensed, underscoring the important role continuing education plays after licensure. Additionally, architects working in smaller firms rated their typical level of knowledge/skill use at “evaluate” more frequently than those working in medium and larger firms, reinforcing that the small-firm practitioner is typically responsible for performing a broader range of tasks in their daily work.

Further analysis and application of Practice Analysis data will help ensure the ARE remains psychometrically justifiable, legally defensible, and relevant to current practice.

Use and Application

The 2012 NCARB Practice Analysis has already had a meaningful influence on the immediate future of the ARE as a guide for refreshing the existing exam database. The survey’s ongoing impact will be seen throughout its application over the next few years, as the Council continues to explore new means and methods for examination development and delivery. In addition, survey results will also inform future versions of the ARE. A supplement to the Examination Report, further identifying the long-term application of the Practice Analysis data, will be released in early 2014 following further research and analysis by various NCARB committees and task forces.

Background

A practice analysis has historically been conducted by NCARB every five to seven years. The primary goal of previous studies was to gather data for purposes of maintaining a current and valid test specification for the Architect Registration Examination® (ARE®). The Council expanded the scope of the 2012 study, adding new rating scales that would answer various research questions pertinent to education, internship, examination, and continuing education. As a result, the survey design, data collection, data analysis, and application processes were significantly revamped. As in the past, the Practice Analysis is consistent with the benchmark Standards for Educational and Psychological Testing. The 2012 NCARB Practice Analysis of Architecture was designed under the guidance and review of a Practice Analysis Steering Committee (PASC) comprised of Member Board Members and architects representing the profession’s collateral organizations: the Association of Collegiate Schools of Architecture (ACSA), the American Institute of Architects (AIA), the American Institute of Architecture Students (AIAS), and the National Architectural Accrediting Board (NAAB). NCARB selected PSI Services, LLC, to conduct the study, which included a total of 11 separate surveys in order to decrease the amount of time required to complete the survey and to help ensure that a statistically valid response rate would be obtained. The final analysis sample included 7,800 responses, providing a substantive basis for summarizing professional practice through its representativeness, statistical precision, and breadth of information.

The complete 2012 NCARB Practice Analysis of Architecture will be released in mid-June 2013, following the release of the individual reports. Please contact Amanda Pica at apica@ncarb.org if interested in reproducing any of the information included in the Examination Report.

######

About NCARB

The National Council of Architectural Registration Boards’ membership is made up of the architectural registration boards of all 50 states as well as those of the District of Columbia, Puerto Rico, Guam, and the U.S. Virgin Islands. NCARB assists its Member Boards in carrying out their duties and provides a certification program for individual architects.

NCARB protects the public health, safety, and welfare by leading the regulation of the practice of architecture through the development and application of standards for licensure and credentialing of architects. In order to achieve these goals, the Council develops and recommends standards to be required of an applicant for architectural registration; develops and recommends standards regulating the practice of architecture; provides to Member Boards a process for certifying the qualifications of an architect for registration; and represents the interests of Member Boards before public and private agencies. NCARB has established reciprocal registration for architects in the United States and Canada.

Visit: www.ncarb.org
Twitter: www.twitter.com/ncarb
Facebook: www.facebook.com/NCARB
YouTube: www.youtube.com/NCARBorg

Related Content

Practice Analysis of Architecture
NCARB’s Practice Analysis of Architecture survey, conducted every five to seven years, provides essential insight into the practice of architecture. Findings are significant to the profession and help determine the knowledge and skills necessary to practice architecture independently and protect the public’s health, safety, and welfare.
[more]


Practice Analysis of Architecture: Examination Report
For the first time in the history of architecture practice analyses, architects were asked “How frequently have you performed the task during the past year?”
[more]


Internship Report Reveals Key Insights From NCARB Practice Analysis
The Internship Report presents data and key insights that will allow the Council to undertake a fresh and comprehensive review of how architectural internship serves as an important bridge connecting education with licensure.
[more]


Education Report Reveals Key Insights from NCARB Practice Analysis
A series of special reports from NCARB will share education, internship, examination, and continuing education-related findings from its 2012 Practice Analysis of Architecture. The Education Report is now available to download and presents data and key insights relevant to architectural education.
[more]