April 15th Network Expansion Information Session - Questions and Answers

On April 15th, 2011 the CIRTL Network hosted an online Q&A informational session for institutions interested in applying for membership in the expanded CIRTL Network. Many of the questions and answers from April 15th session appear below. Please contact Kitch Barnicle (608-262-9174, kabarnicle@wisc.edu) if you have additional questions.

If you have a question that is not addressed here or on the Expansion FAQ page you may submit it here.

Return to the Expansion page 

1.) Can you provide an example of the required annual evaluation report (and any other reporting requirements)?

In our evaluation approach we model Teaching-as-Research. Each partner institution conducts evaluation research on CIRTL programs, including assessing the impact of those programs. The list below summarizes our typical reporting requirements. 

1. Each year partner institutions submit a scope of work (SOW) as part of the subcontract process. The 4-6 page SOW covers goals, program offerings, cross-network offerings, management structure and budget justification. The SOW is also expected to include an institutional evaluation plan.
2. Institutions are also required to submit an annual Institutional Portrait. The portrait is a template set of questions that institutions use to describe their programs and results for the year as well as their plans for the coming year. Download: Institutional Portrait Template (PDF)  Institutional Portrait Demographic Worksheet (PDF)
3. Institutions are required to submit an evaluation report for one or more of their local CIRTL offerings (ideally for each of their CIRTL offerings). The complexity of these evaluations will vary across programs.
4. Each year institutions are asked to submit a 1-2 page update for inclusion in the annual report that goes to NSF and the National Advisory Board. Typically, these updates pull material from the SOW, institutional portrait and evaluation reports.
5. Evaluation is the responsibility of the CIRTL project leader at each institution. Network institutions typically use some of their CIRTL resources for their evaluation efforts. At some institutions, a graduate student has been employed to work on program evaluations as a member of the campus CIRTL team.

Continual program evaluation and improvement is a key to how CIRTL operates. The CIRTL evaluation team helps program facilitators choose the appropriate evaluation approach to fit the program, as well as the time and resources available to conduct the evaluation. Athough the above reflects our current evaluation reporting requirements, just as CIRTL programs continue to evolve, so too does our evaluation process.

As an example, current CIRTL community members are in the process of completing summative evaluation reports of their programs based on the questions in this Evaluation Report form template (PDF).

2.) By signing on to the expanded network, given there are not commitments by any funding source, what is the financial implication for universities in the network? Is the expectation that CIRTL activity & costs will be incurred by participating institutions before external funding is secured, or only once external resources are secured? (I understand 5 years out there is an expectation for institutional financial responsibility.)

We are working on a success plan here. We have had good success with our external funding, CIRTL is well-known at NSF and our expectation is that we will be successful in this process.

With that said, NSF recently took a 10% cut and the landscape of funding has certainly changed. So we are doing several things in order to diversify the external funding portfolio. We have started conversations with NIH, DOE and possibly NASA, in order to diversify the funding. We are also looking into private foundations.

We're not making guarantees, but we are doing our best to have success here. But should we not have success we will have to step back and decide how we will move forward in what will clearly be a substantially less funded domain. We probably would have to accelerate the dues plan. We would need to scale back our operating plan, and we would need to resubmit.

The other part of the question is one that I don’t have an answer for yet, and that is "when do we start"? The money isn’t going to flow until roughly June 2012, but we will have the Network together in October. We’re going to start an active conversation Monday, and certainly in October, on what we can do in the meantime. I would imagine that the schools will want to be actively moving forward on how they plan to use their resources.

3.) What are you looking for in a letter of intent? A desire to participate or something more complicated? How many institutions have you invited to apply?

The Letter of Intent is for us to know what we need to do to prepare for the review process based on how many applications we expect. We are not going to be evaluating you on the letter of intent. The form simply asks for contact details and welcomes comments.

We have sent out application packages to the graduate deans and provosts of the 100 largest (STEM) Ph.D. producers in the country. That was our broad sample of “cold calls.” And we have sent out roughly another 30 to people we’ve met at meetings.

4.) A couple of questions on the outcome. What can you say about where CIRTL students go and what success they have in getting jobs? Are they going to R1s? Is there a drive to put them there? Or is it simply to prepare the best educated students with the ability to go into any of the types of positions?

Our motivation is very much the latter. We are trying to prepare students so that they can find the careers where they can be effective. We would be concerned if none of the students went to R1s. This would be reinforcing current cultural thinking that we’re trying to change. Remember that roughly 10-15% of grad students at R1s end up teaching at R1s. So the placement of CIRTL Network program participants will reflect this, and most will end up going to institutions other than R1s.

We have a longitudinal study underway supported by NSF to follow roughly 100 of the students that have been through the prototype learning community, as well as students that have been through future faculty prep programs at ASU and University of Washington.

In terms of convincing your own faculty, that depends on the institution itself -- its culture and what it values. We have got quite a bit of feedback from students and, ultimately, their faculty, that they have had substantial success in getting jobs because they are prepared to answer the question intelligently, "how are you going to teach"? That’s not just true at Liberal Arts universities. Research is primary at all institutions, but when you’re down to the final five job candidates, the person who can also talk intelligently about teaching has a leg up in getting hired.

5.) Are you working with the Council of Graduate Schools on this effort?

At best, indirectly. I think that’s going to change, but so far we’ve been finding out about each other through the expansion process. We’d like to enhance that in the coming years, as well as with disciplinary societies.

6.) What are the characteristics of a successful application? -- What are you looking for?

Criteria for Review of Applications for Membership in the CIRTL National Network

7.) We don’t have a Center for Teaching and Learning, but we do have a department of Teaching & Learning in our College of Education, and there are a number of faculty that are involved in STEM education. Can I include them?

Absolutely. We collaborate closely with our school of education here at Wisconsin. I would caution the proposal being housed in the school of education. It doesn’t tend to go well with STEM faculty when the programs are not coming out of STEM departments.

8.) I noticed that some of these six schools have graduate school officials as part of the leadership team. Does the graduate school need to be involved with the application?
It’s not necessary but it certainly is a positive. You do need to make the case that you have administrative support for what you’re trying to do.

9.) How much are you looking for each of the institutions to come up with their own new types of learning community activities as opposed to adapting successful CIRTL programs?

 We have seen the full spectrum in the prototype. Others have developed their own programs and integrated the CIRTL Pillars into what they’re already doing. We’re looking for a plan that will succeed. We’re more than happy to provide anything and everything you might need to do cold start, and just are excited to hear of new approaches.

10.) In terms of evaluation, are you interested in social network analysis, time course development of ego-nets, instruments which measure collaborations more directly?

I would reiterate that each institution should feel free to use what methods of evaluation are appropriate for their programs and institutional goals. Most of our institutions are not implementing evaluation methods that are quite at the level of sophistication as those mentioned in the question, but we always encourage institutions to implement those methods that they feel are appropriate. There are also opportunities for supplement grants for evaluation purposes.

11.) We understand that the focus should be centrally on graduate students, but certainly this group of students is affected by the faculty and affects the undergraduate student population. What proportion of designed activities would you like to see, or do you typically see allocated to the training, etc. of graduate students, undergraduate students, and/or faculty/post-docs? 50/25/25? (Essentially, who is it "okay" to serve?)

When we say graduate students we mean graduate students and post-docs. Post-docs are a critical component of future faculty, they are the new frontier of what we’re trying to do. Ideas that are innovative in terms of integrating post-docs into the programs are very much desired.

We appreciate that there is a pipeline from undergrad to faculty. We are trying to do our best in our domain (grad students & post-docs) and collaborate with those who address the needs at other stages of the pipeline.

We have also experienced what we have come a "trickle-up effect." That is, the more faculty you engage, the more faculty development  will occur both in terms of mind-set change and what they can do in their teaching.

12.) The history of CIRTL is focused a lot around biology, chemistry, physics, and some engineering. How are you going to bring engineering more into the fold? Are you targeting institutions with large engineering programs?

Throughout CIRTL's history, we have been focused cross-STEM all the way. Engineering has been as engaged as any other field. That said, we would like to have some engineering-focused schools in the Network. We are very interested to see how these ideas play at institutions that have particularly targeted populations.

13.) Can nursing, public health, and medical students in graduate students be included? What about doctoral (PhD) programs in nursing?

Medical students do not participate in our programs. However, graduate students pursuing PhDs within a medical or nursing school can, and do, participate in our programs at UW-Madison (e.g physiology, medical physics, neuroscience). We try to define STEM as broadly as we can because we are trying to have the broadest impact. The key is preparing future faculty, so we are talking about preparing students in those fields to become faculty. Nursing has been involved in CIRTL at UW-Madison.

14.) Given your evaluation criteria, will you be placing priority on selecting institutions that produce the most PhDs, or would this issue be another example of how numbers are not the entire issue, as you spoke about earlier?

I think that’s exactly what I meant to be speaking to. Numbers matter, but if someone is bringing to the Network an important niche, either in where their students go as faculty or the way they can prepare a large number of faculty across the Network to be better faculty, that’s the kind of diversity that we are weighing very highly.

15.) You mentioned that the programs should not be housed in the College of Education. At our institution, we have a long-standing rich history of collaborations with Engineering. Could you speak about how our Leaders and Co-Leaders might be distributed? Should the Leader come out of a STEM department and a Co-Leader from the College of Ed?

Each campus has its own culture, so particular organizational arrangements should be decided based on what is most appropriate for each institution. Generally, however, we have found that, since CIRTL focuses on the preparation of STEM doctoral students for their careers, locating the programs in relationship to a STEM college or within a Graduate School that serves students across disciplines is most effective. Leadership by a STEM-related faculty member lends credibility to the program for both student participants and their STEM faculty advisors. Furthermore, a STEM faculty member typically has particular insights into the issues, interests, and challenges facing STEM doctoral students and post-docs. While leadership provided by STEM faculty has been important at current participating institutions, involving co-leaders with expertise in education has also enriched CIRTL programs. For example, Ann Austin, a CIRTL co-PI is from the College of Education at Michigan State University.

16.) For the 5 programs you mention in the application (Q5), do you want five doctoral degree-granting programs in STEM or five support units (such as grad school, center for teaching and learning, etc.)? Are those doctoral-degree programs or preparing future-faculty programs?

The latter. We are trying to gauge the current landscape with respect to the preparation of doctoral students around teaching and learning at institutions applying for membership. If you do not have existing programs, that’s okay.

17.) Would it make sense to also include PhD-granting programs on the medical campus, especially as you are looking to expand the funding stream to include NIH in particular as well as other agencies?

 We have been having active conversations with our medical school to see to what extent we can integrate the basic biomedical science graduate programs into our programs. We’ve already done it with nursing. The team at Vanderbilt University has done work with basic biomedical science programs to some extent but I think it could be done much more.

18.) For question number 2 on the membership application, about STEM doctoral programs that will participate, there is not a limit of 5, correct?

Correct, there is no limit on the number of STEM doctoral programs that can participate in CIRTL on your campus.

19.) With respect to the sample budget presented, how tightly are you expecting us to hold to that sample? Is there flexibility if it is going to take more resources?

In terms of the cash flow and allocation, the budget in the Growth Plan is meant to be a sample. In terms of total amount, the figures in the sample budget are more than what the current Network have received thus far, though not by much. The total amount of funding available to member institutions for the base operations of the learning community is not likely to exceed the amounts in the sample budget. However, there is flexibility in terms of how those funds are used.