10 June 2013
Washington, DC—The Examination Report, the third in a special series of publications from the National Council of Architectural Registration Boards (NCARB), shares examination-related findings from the most comprehensive practice analysis ever undertaken for the architecture profession. Required by all 54 U.S. jurisdictions, the ARE plays a critical role in assessing the knowledge, skills, and abilities to provide the various services required for the independent practice of architecture.
“In addition to informing interim updates to the current version of the ARE, the Practice Analysis will serve as a foundation for the development of future versions of the exam,” said NCARB CEO Michael J. Armstrong. “The findings will also have a significant impact on our exploration of alternative pathways to licensure that further blend the three traditional components of education, internship, and examination.”
The Examination Report encompasses extensive data collected from three examination-specific surveys:
- Architects were asked to indicate how frequently they performed the task in the past year and to rate the level of importance of the competent performance of the task by a recently licensed architect practicing independently;
- In this survey, architects were presented with two similar but distinct questions. The first question is very common in practice analyses and asks when each knowledge/skill was acquired. The second question asked the same respondents to identify when each knowledge/skill should be acquired; and
- In the third survey, architects were asked to rate the importance of each knowledge/skill to a recently licensed architect practicing independently and at what level they typically use the knowledge/skill when performing their job.
Highlights of Examination Findings
The Practice Analysis provided an opportunity to analyze the ARE in relation to the contemporary practice of architecture. The resulting findings were the product of five key areas of analysis:
- Level of Importance – The survey indicates that 129 of the 132 knowledge/skills, and 106 of the 110 tasks, were rated as “important” or greater by architects who completed the survey. Three of these K/S and tasks were rated as “critically important” and are directly related to the protection of the public health, safety, and welfare.
- Point of Knowledge/Skill Acquisition – When comparing level of importance with point of acquisition, 15 knowledge/skills were identified as “important” or greater and also identified as being acquired after licensure by more than 50 percent of architects completing the survey. These 15 knowledge/skills primarily deal with practice and project management issues and are vital to competent practice; therefore, their acquisition should be better supported during education and internship.
- Level of Knowledge/Skill Use – Architects were asked to rate the level at which they use each knowledge/skill. “Apply” was the most frequently selected response at 42.5 percent. “Evaluate” and “Understand” were evenly split at 26.0 percent and 25.7 percent, respectively. Only 5.8 percent of architects indicated they did not use the knowledge/skill in their job. This data will be used to support item writers in the creation of more relevant items/questions for the examination.
- Frequency of Task Performance – Over 70 percent of the tasks included in the survey were indicated by architects as being performed in the past year. Most tasks were rated as being performed “quarterly” (20.4 percent) or “monthly” (19.0 percent). This data will be used to refine the content and distribution of items included in the ARE. Ten tasks rated “important” or greater were identified as “not performed” in the past year by more than 50 percent of architects. Additional analysis by various NCARB committees is warranted to better understand the nature of those tasks.
- Sub-group Analysis – Response rates based on years of licensed experience and size of firm revealed little variation. More experienced practitioners tended to report a slightly higher level of ability than those recently licensed, underscoring the important role continuing education plays after licensure. Additionally, architects working in smaller firms rated their typical level of knowledge/skill use at “evaluate” more frequently than those working in medium and larger firms, reinforcing that the small-firm practitioner is typically responsible for performing a broader range of tasks in their daily work.
Further analysis and application of Practice Analysis data will help ensure the ARE remains psychometrically justifiable, legally defensible, and relevant to current practice.
Use and Application
The 2012 NCARB Practice Analysis has already had a meaningful influence on the immediate future of the ARE as a guide for refreshing the existing exam database. The survey’s ongoing impact will be seen throughout its application over the next few years, as the Council continues to explore new means and methods for examination development and delivery. In addition, survey results will also inform future versions of the ARE. A supplement to the Examination Report, further identifying the long-term application of the Practice Analysis data, will be released in early 2014 following further research and analysis by various NCARB committees and task forces.
A practice analysis has historically been conducted by NCARB every five to seven years. The primary goal of previous studies was to gather data for purposes of maintaining a current and valid test specification for the Architect Registration Examination® (ARE®). The Council expanded the scope of the 2012 study, adding new rating scales that would answer various research questions pertinent to education, internship, examination, and continuing education. As a result, the survey design, data collection, data analysis, and application processes were significantly revamped. As in the past, the Practice Analysis is consistent with the benchmark Standards for Educational and Psychological Testing. The 2012 NCARB Practice Analysis of Architecture was designed under the guidance and review of a Practice Analysis Steering Committee (PASC) comprised of Member Board Members and architects representing the profession’s collateral organizations: the Association of Collegiate Schools of Architecture (ACSA), the American Institute of Architects (AIA), the American Institute of Architecture Students (AIAS), and the National Architectural Accrediting Board (NAAB). NCARB selected PSI Services, LLC, to conduct the study, which included a total of 11 separate surveys in order to decrease the amount of time required to complete the survey and to help ensure that a statistically valid response rate would be obtained. The final analysis sample included 7,800 responses, providing a substantive basis for summarizing professional practice through its representativeness, statistical precision, and breadth of information.
The complete 2012 NCARB Practice Analysis of Architecture will be released in mid-June 2013, following the release of the individual reports. Please contact Amanda Pica at email@example.com if interested in reproducing any of the information included in the Examination Report.
The National Council of Architectural Registration Boards’ membership is made up of the architectural registration boards of all 50 states as well as those of the District of Columbia, Puerto Rico, Guam, and the U.S. Virgin Islands. NCARB assists its Member Boards in carrying out their duties and provides a certification program for individual architects.
NCARB protects the public health, safety, and welfare by leading the regulation of the practice of architecture through the development and application of standards for licensure and credentialing of architects. In order to achieve these goals, the Council develops and recommends standards to be required of an applicant for architectural registration; develops and recommends standards regulating the practice of architecture; provides to Member Boards a process for certifying the qualifications of an architect for registration; and represents the interests of Member Boards before public and private agencies. NCARB has established reciprocal registration for architects in the United States and Canada.