The Critical Review is a Brown University student organization that publishes reviews of undergraduate courses, based on responses from students and instructors to questionnaires. The organization, which began in 1976 as a publication of the Undergraduate Council of Students, is now recognized by UCS as a Category III Student Group. The Critical Review is the only organization at Brown that produces written and numerical course evaluations that are available publicly and that are designed to reflect the consensus of the students in each class.
The Critical Review has two primary objectives in serving the Brown community:
Each semester, the Critical Review distributes questionnaires to the instructor of each undergraduate course, except for independent studies. Instructors distribute the questionnaires to their students during the last days of the semester and ask one of their students to collect the questionnaires and return them to the Critical Review.
Critical Review writers then use customized software and the Critical Review writer's guide to compose a review for each course. The course evaluations are based on just the comments on the returned questionnaires and are designed to reflect the consensus of the students in each class. These materials are then handed over to the editors, who edit the writing and create the page layouts that form the Critical Review magazine. After final editing, the finished magazine is printed and the reviews are also published online.
All courses that return a sufficient number of completed surveys are reviewed. All reviews are checked numerous times for accuracy and objectivity: the Critical Review does not tolerate bias in the writing process. The reviews are written, edited, and published by students; the organization is managerially and financially independent of the university itself. Funding is provided by the Undergraduate Finance Board.
The questions on the Critical Review questionnaire are the final product of extensive discussion between the members of the organization and the Brown faculty and student body. The questionnaire has undergone comprehensive revision over the decades, always with the input of both students and faculty members. The Critical Review editors have also looked at research on the validity of student ratings of instruction and have examined evaluation forms used by various departments at Brown and other colleges in trying to determine how to design and improve the questionnaire.
The questions are designed to elicit responses from students and instructors which are as constructive and as informative as possible. The organization makes every effort to present facts about the way a course was structured and taught that explain why students felt the way they did about it. We try not to emphasize individual instructors' styles or popularity and instead focus on measurable aspects of good instruction: whether goals and expectations were clear, whether and how students were empowered to get as much as they wanted out of a course, whether students were able to participate and receive feedback regularly, etc.
Many faculty members state that the Critical Review questionnaire is already superior to many of Brown's departmental evaluation forms, but just as we believe that all courses are capable of being improved, we believe that no questionnaire is perfect either. We encourage and welcome suggestions for ways to further improve our questionnaires.
The bar graphs that accompany the written reviews in the Critical Review present all of the responses that students provided to the numerical rating questions on the questionnaires. Each questionnaire statement is summarized on the left, and the bar on the right indicates the percentage of the class that gave each response. The size of the dark blue portion of the bar indicates the fraction of the class that responded "1" or "strongly agree" with this statement; the next segment of the bar represents the number of students who gave a response of "2;" etc. The green segment represents students who gave a response of "4" or "strongly disagree." Some bars do not reach as far to the right as others. The blank space to the right of a bar represents students who gave a "Not Applicable" response, meaning that the particular statement did not apply for the course. This fraction also includes students who did not reply to the question.
The instructor average is the average response given to questions in the instructor section of the questionnaire. The course average is the average response given to the questions in the Structure, Content, and Skills sections of the questionnaire. The averages should not be interpreted as an absolute method of judging courses. They will not reflect the experience of every student in a particular course or with a particular professor. The degree of variation in the responses provided by students, as evident from the bar graph itself, is much more important than the averages in determining the degree of student satisfaction with a particular course.
The Critical Review publishes a review of every course that returns a sufficient number of completed questionnaires. The total number of enrolled students (according to the Registrar) and the number of respondents are noted and presented with each review, for context.
The following types of classes are not reviewed for the reasons provided:
Participation in the Critical Review is optional for instructors. Some instructors choose not to distribute the questionnaires to their students, so we are unable to publish reviews of their classes. We ask all students who wish for their courses to be reviewed to tell their instructors how much their participation would be appreciated. If you know of a particular instructor who has chosen not to participate in the Critical Review, please contact us at Critical_Review@brown.edu. We'd love the opportunity to speak with him or her so that we can find out why he or she chooses not to participate.
Several Critical Review readers and even some instructors have advocated making the Critical Review questionnaire available online. One reason for this is that with such a system, instructors would no longer need to sacrifice class time for the questionnaires to be filled out because students would be able to do it on their own time. Another thought is that more courses could be reviewed because participation would no longer be optional for instructors under such an arrangement. Some have even pointed out that such a system would be more environmentally friendly because the Critical Review would no longer need to print so many questionnaires every semester.
However, there are many strong arguments against making the Critical Review questionnaire available in some type of online submission system. The obvious concerns are that there would be serious security concerns regarding submitted responses and that such a system would be a lot of work to establish in the first place. There is also the problem of human resources and limited production time: at present, the Critical Review would not have the capacity to produce a review for every single undergraduate course.
The less obvious implications are even more important, however. With an online questionnaire, even if it was completely secure, the reviews themselves would almost certainly change fundamentally. As evidenced by the fact that schools with online course evaluation surveys tend to receive very positive comments, very negative comments, and little in between, there is no reason to believe that the Critical Review would receive adequate samples of each class to represent the consensus of the students, and thus the reviews would be utterly invalid and useless. Quite simply, most students who feel indifferent about a course believe they have little to say, and are unlikely to take the time to respond to a questionnaire unless it is physically handed to them.
We try to complete each new edition of the Critical Review by the beginning of preregistration period. This means that in the autumn the Critical Review comes out in mid-October, and in the spring the new edition comes out in mid-April. The precise date that the new edition will be available online can be found on the Critical Review web site shortly before preregistration begins.
You can browse the latest version of the Critical Review by visiting our website, http://www.thecriticalreview.org. The Critical Review no longer prints and distributes paper copies for free.
Coming soon.
The Critical Review distributes packets of questionnaires to academic departments a few weeks before reading period begins. We ask administrative assistants in each department to distribute the packets to the individual instructors.
We handle a large volume of packets of questionnaires, and occasionally a few of them are accidentally delivered to the wrong department. Some of them are accidentally returned to us blank by the departments immediately after they are delivered. If you have not received questionnaires for your classes by two days after the announced delivery date, check with your department's administrative assistants to find out whether they have already distributed all packets to instructors. If they have not received them, or if they have received and distributed them but you did not receive one for your class, please contact us at Critical_Review@brown.edu so that we can locate and deliver your packet of questionnaires or bring you a new one as soon as possible.
Critical Review prefers instructors to use the official questionnaires, as they have the course details pre-filled and come in pre-addressed, barcoded envelopes. However, if you cannot wait for the Critical Review to deliver questionnaires, you can print the following generic questionnaires yourself:
Please return the completed questionnaires via Campus Mail to Critical Review, Box 1930.
Coming soon
The Critical Review is always looking for new staff members. All training is on-the-job and the time requirement is relatively low, as there are only a few required meetings each semester and writers can prepare most of their reviews on their own time. If you are interested, please e-mail us at Critical_Review@brown.edu for more information about how you can get involved!
Only members of the Brown community can read reviews. That means you must visit from a computer on Brown's network, or if visiting from off-campus, log in with your Brown password.