Value Added Measurement: Wanted Professional Palm Reader

Want to bet on a horse? Put your trust in a bookie whose job it is to predict winners after studying potential to win based on breeding, training, and standings in recent races. Want to invest in the stockmarket? Put your trust in a stockbroker whose job it is to predict companies that show signs of being a good investment, with potential growth, and a stable financial structure.

Want to know about a child's school achievement. Put your trust in an algorithm, a formula which will predict how much progress the student should make based on a complicated equation of 10 factors, but do not ask how it works. Florida determined that socio-economics would not be included as one of the 10 predicting factors.

StateImpact Florida and the Miami Herald went looking for some explanation on how it will work and they got this answer:

"No lay person, teacher or reporter can understand it. So just trust us."

This formula will be used in Florida as the basis for merit pay this way:

The formula is designed to predict how students will score on the state’s standardized exam—the FCAT. And then it adjusts teachers’ pay depending on how well their students measure up against that predicted score.

Until recently, for $190 Chinese parents signed their children up for "palm-reading tests that could allegedly tell a child's intelligence and professional aptitude." Palm-reading tests have been determined to be pseudoscience and Chinese educational authorities banned the practice.

"Predicting is not an equation."

Related posts:
SB736/HB7019: The trouble with value-added measurement
NUT Report: "We have to do something."
Student Data Collection: Purpose, Costs, Risks?

Education Reform: Show me the money - still asking

More and more fact-based, smart, and talented parents, community members, and taxpayers are blogging on education reform initiatives, filling in a notable gap and spurring a better informed public on the current initiatives.

Grumpy Educators recommends Race to the Top is a Race Off a Cliff posted on the Seattle Education blog.

Education reformers like to use the words "disruptive innovation" to describe current initiatives. I see a lot of expensive disruption and little innovation.

Seattle Education details on the costly confusion and waste that Race to the Top is causing.

Speed bump: New teacher evaluation process in Brevard County, Florida

A front page story in the paper edition of the Florida Today caught my eye today in the supermarket. "Teaching tool or trouble brewing?" describes the implementation of the new requirements for teacher evaluation in Brevard County, Florida schools. By 2014, this evaluation system will be used to create a merit pay system, which will affect teacher pay in accordance with recent legislation. So far, the article has not appeared in electronic format.

According to the article, "school board members have heard that the evaluation process is not being consistently implemented in the schools - leading to frustration and confusion among some of the 5,000 teachers." The process speed bump seems to center on the teacher-written professional growth plan, which accounts for 50% of the evaluation procedure. Teachers in the district report different directions on how to proceed with professional growth plans, apparently at the school level. School district official, Joy Salamone, said the district would fix the issues if the union provided specifics.

The Florida Today article described at least two specific incidents in adequate fashion for the district to act. The new process is a dramatic change, so confusion can be expected. While teachers and administrators should be focused on school students, their attention is diverted to non-instructional tasks. Let's hope the school district can create and maintain an atmosphere where confusions can be clarified without fear and without substantial time taken from classroom instruction.

The other 50% of teacher evaluation will be based on student performance results on FCAT reading scores or standardized end-of-course exams. It is still unclear, at least to this community member, how student performance will apply to P.E. teachers and special education teachers and how the algorithm will be developed.

It is refreshing to see the Florida Today do some reporting on educational issues and hope they do more.

Reference: Ryan, Mackenzie, "Teaching tool or trouble brewing?," Florida Today, October 1, 2011.

Goofy on teachers in 1952

North Carolina student data revealed online: State Longitudinal Data Systems Revisited

A phone call from a parent alerted North Carolina Department of Public Instruction officials that private data on school students was posted online.

"Data housed on a N.C. State University computer server that contained private information for about 1,800 schoolchildren from Richmond and Wilson counties was inadvertently made available online, university officials said Tuesday.

The data, gathered from 2003 to 2006 as part of a research study on classroom practices, included names, Social Security numbers and dates of birth. The three affected elementary schools are the now-closed Ashley Chapel in Richmond County and Gardners and Wells in Wilson County."

Read more on this breach of data here.

Although the source of the error is not described, the news report alludes to a computer "glitch" of some type. How seriously should this breach be taken? Could student data be hacked, repackaged, sold, and revealed?

Recent reports reveal an Anonymous group of overseas hackers have threatened attacks on U.S. law enforcement computer systems. Of the 70 attacks claimed by the group, only Arkansas and Louisiana attacks have been verified. Officials say that no sensitive data was accessed.

Last month the U.S. National Security Council released a report in which it named cyber crime as a major threat to national security, and costing the U.S. $1 billion annually in losses.

With Race to the Top dollars, states have been developing State Longitudinal Data Systems using federal parameters and guidance. However, the Los Angeles Times reports that California's Governor Brown recently decided to reverse that State's decision in development of the SLDS saying districts had the data they needed.

In a February 2010 letter, U.S. House Representative John Kline wrote Secretary of Education Arne Duncan last year regarding this initiative:

"As part of what you described as a "cradle to career agenda," the Department of Education is aggressively moving to expand data system that collect information on our nation's students. I am concerned by recent reports that indicate the Department's hasty pursuit of this goal could compromise student privacy rights."

Kline goes on to say: "The Department's efforts to shepherd states toward the creation of a de facto national student database raises serious legal and prudential questions. Congress has never authorized the Department of Education to facilitate the creation of a national student database."

Read Rep. Kline's full letter here.

Teacher performance pay is described as the rationale for this massive data collection initiative. Does a performance pay structure through data collection outweigh the costs, risks, and privacy considerations?

For facts and details on SLDS, go to Truth in American Education.

SB736/HB7019: The trouble with value-added measurement

SB736/HB7019 intends to apply value-added measurement (VAM) to connect student achievement on a test as a performance indicator of the teacher, or a poor student score = poor teaching. VAM uses statistical tools to calculate the contributions a teacher makes to student achievement gains. The formula for the calculation includes factors the developer chooses to include. What do experts have to say about the value of value-added measurement?

1) Jim Angermeyr, Director of Research, Evaluation & Testing for Bloomington Public Schools and "one of the designers of a widely respected value-added test lots of Minnesota schoolchildren take two or three times a year" was interviewed by Beth Hawkins. In the article, "Do 'value-added' teacher data really add value?", Angermeyr is described as "something of a standardized testing skeptic. He believes that economists tend to believe in using value-added data in evaluation. Educators and psychometricians, not so much."

“It’s not necessarily that the methodologies are wrong,” he said. “It’s that the inferences we’re drawing can be wrong.”

"The kids are the greatest of the variables, of course. The tests may tell you a student is reading better or sliding in math, but they don’t tell you whether she spent the summer with a tutor or he is so young the test isn’t as accurate as it would be in an older child.

Nor is the same test used from year to year. A particular student or teacher may fare better on a test closely normed with curriculum vs. one aligned with a set of knowledge-based standards."

“You leave out a lot of the potential variables,” Angermeyr said.
“They’re just not at the point where we should use them to make decisions about jobs.”

2) The National Research Council and the National Academy of Education gathered experts in the this field and held a workshop titled "Getting Value Out of Value Added." There was general agreement by participants that there is no single statistical model and still a work in progress for applicability when measuring teacher quality.

Discussion resulted in these additional conclusions:

• Results generated by existing models have a high degree of instability.
• Good results require good tests and good test results.
• Experts find it useful for low stakes use to identify areas that need improvement.
• Experts recommend against using it for high stakes such as, teacher pay.

According to Henry Braun, the group determined the following:

"To nobody’s surprise, there is not one dominant VAM. Each major class of models has shortcomings, there is no consensus on the best approaches, and little work has been done on synthesizing the best aspects of each approach. There are questions about the accuracy and stability of value-added estimates of schools, teachers, or program effects. More needs to be learned about how these properties differ, using different value-added techniques and under different conditions. Most of the workshop participants argued that steps need to be taken to improve accuracy if the estimates are to be used as a primary indicator for high-stakes decisions; rather, value-added estimates should best be used in combination with other indicators. But most thought that the degree of precision and stability does seem sufficient to justify low-stakes uses of value-added results for research, evaluation, or improvement when there are no serious consequences for individual teachers, administrators, or students." (p.54)

While Florida plunges into creating dozens of new tests, North Carolina's legislature in bipartisan agreement is sending a bill to the Governor to end some end-of-course tests, maintaining those to meet federal requirements and to measure student achievement. Two years ago, they voted to end a few others. This legislature wishes to stop paying for so many tests that are both expensive and failing to yield the returns once thought beneficial.

Concerns over the costs for implementing SB736/HB7019 by Florida Senators and by school boards are being reported. In the end, this is an unfunded mandate using an experimental statistical model, and an expensive test development and implementation scheme that extends far beyond the reach of RT3 dollars. Legislators remain silent on the issue of costs and cost benefits. The public has a right to know.

To read the "Getting Value Out Of Value Added" report for free, go to the widget on the right side of this page, select the icon that looks like an open book with the word Read underneath, and then Open Book in the small screen area. The document will open so you can read it easily.

The experts in the National Research Council and the National Academy of Education in this conference included:

Rita Ahrens, Education Policy Studies
Joan Auchter, National Board for Professional Teaching Standards
Terri Baker, Center for Education, The National Academies
Dale Ballou, Vanderbilt University
Henry Braun, Boston College
Derek Briggs, University of Colorado at Boulder
Tom Broitman, National Board for Professional Teaching Standards
Alice Cain, House Committee on Education and Labor
Duncan Chaplin, Mathematica Policy Research
Naomi Chudowsky, Center for Education, The National Academies
Pat DeVito, AE Concepts
Beverly Donohue, New Visions for Public Schools
Karen Douglas, International Reading Association
Kelly Duncan, Center for Education, The National Academies
John Q. Easton, Consortium on Chicago School Research
Stuart W. Elliott, Center for Education, The National Academies
Maria Ferrão, Universidade da Beira Interior, Portugal
Rebecca Fitch, Office of Civil Rights
Shannon Fox, National Board for Professional Teaching Standards
Jianbin Fu, Educational Testing Service
Adam Gamoran, University of Wisconsin–Madison
Karen Golembeski, National Association for Learning Disabilities
Robert Gordon, Center for American Progress
Jeffrey Grigg, University of Wisconsin
Victoria Hammer, Department of Education
Jane Hannaway, Education Policy Center
Patricia Harvey, Center for Education, The National Academies
Lloyd Horwich, House Committee on Education and Labor
Lindsey Hunsicker, Committee on Health, Education, Labor, and Pensions
Ben Jensen, Organisation for Economic Co-operation and Development
Ashish Jha, Harvard School of Public Health
Moshe Justman, Ben Gurion University, Israel
Laura Kaloi, National Association for Learning Disabilities
Michael Kane, National Conference of Bar Examiners
Judith Koenig, Center for Education, The National Academies
Michael J. Kolen, University of Iowa
Adam Korobow, LMI Research Institute
Helen F. Ladd, Duke University
Kevin Lang, Boston University
Sharon Lewis, House Committee on Education and Labor
Valerie Link, Educational Testing Service
Dane Linn, National Governors Association
Robert L. Linn, University of Colorado at Boulder
J.R. Lockwood, RAND Corporation
Angela Mannici, American Federation of Teachers
Scott Marion, National Center for the Improvement of Educational Assessment
Daniel F. McCaffrey, RAND Corporation
Alexis Miller, LMI Research Institute
Raegen Miller, Center for American Progress
John Papay, Harvard University
Liz Potamites, Mathematica Policy Research
Ali Protik, Mathematica Policy Research
Sean Reardon, Stanford University
Mark D. Reckase, Michigan State University
Andre Rupp, University of Maryland
Sheila Schultz, HumRRO
Lorrie Shepard, University of Colorado at Boulder
Judith Singer, Harvard University
Andrea Solarz, Director of Research Initiatives, National Academy of Education
Gerald Sroufe, American Educational Research Association
Brian Stecher, RAND Corporation
Justin Stone, American Federation of Teachers
David Wakelyn, National Governors Association
Greg White, Executive Director, National Academy of Education
J. Douglas Willms, University of New Brunswick
Mark Wilson, University of California, Berkeley
Laurie Wise, HumRRO

House Committee passes merit pay bill 11-01

The House K-20 Competitiveness Committee passed the House version of a merit pay bill. Minimal differences between the House bill, 11-01, and SB736. Committee Chair Erik Fresen is quoted as saying that "change and reform are scary, but members would look back on today and realize they developed a system that gave each student quality teachers and administrators." I find no information that fiscal impact was discussed. I find no legislative analysis either. The public has a right to know how much this bill will cost above and beyond Race to the Top (RT3) funds, how will those school districts who did not sign on to participate in RT3 be funded or pay for this mandate, and where will the money come from.

Contractors are already on board to help with implementation of the bill.

1) "Houghton Mifflin Harcourt Educational Consulting Services Selected by the Florida Department of Education to Implement a Statewide Teacher and Leader Evaluation System As part of its Race to the Top initiative, the Florida Department of Education, through its partnership with HMH, will provide school districts across the state with a research-based framework and implementation services for a teacher and leader evaluation system."

2) The Center for Teacher Quality has been working with 17 Florida school districts since last year to implement effective evaluation systems. Their website lists some interesting funding partners. In a letter to Florida legislators, the company points to the need for adequate funding.

"If the state and local districts implement the TeacherSolutions, adequate funding must be in place to ensure effective implementation."

"Given our current budget downturn, the state and local districts should consider ways in which to find new dollars to implement these recommendations. Teachers should not be forced to lose compensation in order to fund these ideas."

Adequate funding? New dollars? I haven't heard anything about those items yet.

Meanwhile, Providence, Rhode Island sent lay off notices to all its public school teachers. They are out of money to run the schools due to budget deficits.

UPDATE: The Senate Budget committee voted in favor of SB736, cutting off discussion and debate. No cost analysis found. Senator Haridopolos announced the full Senate will vote on the bill sometime after March 8. I have found no cost analysis.

Missed a blog on SB736 or want to read one again? You will find them all here.

Avatar: www.clipartheaven.com

References:
http://www.teachingquality.org/
http://www.tampabay.com/blogs/the-buzz-florida-politics/content/teacher-merit-pay-bill-passes-first-house-committee
http://www.hmhco.com/news/2011/0217_news_release.html

House Files "Regarding Teacher Quality" Bill

The Florida House announced the filing of the Regarding Teacher Quality Proposed Committee Bill (PCB) on February 16:

For Immediate Release: Wednesday, February 16, 2011 Contact: Lyndsey Cruley, (850) 487-8148 Joint Statement by Representatives Bill Proctor and Erik Fresen Regarding Teacher Quality PCB Filed in the Florida House Tallahassee, Fla. – Representative Bill Proctor (R-St. Augustine), chair of the House Education Committee, and Representative Erik Fresen (R-Miami) chair of the House K-20 Competitiveness Subcommittee and co-sponsor of the proposed committee bill (PCB), today released the following joint statement regarding the filing of the PCB on student success and teacher quality in the Florida House. “It is our belief that every child in our state, regardless of race, creed or economic status, deserves to have an exceptional teacher in the classroom that is vested in their futures and will strive to provide students with a quality education. “This proposed committee bill is another step toward meaningful education reform that will serve to further improve our state’s education system by implementing a performance pay plan that is grounded in student learning growth that will result in the ability to reward the most effective teachers. “We encourage public testimony and input from teachers, parents and administrators as this bill moves through the committee process; and we look forward to the final product reaching the House floor.”

There is no legislative analysis yet. Other reviews of the bill have not yet emerged. I looked over the 44 page bill myself and found that to a high degree, it restates the Senate version of education reform in SB736. Some differences jumped out at me if I understand the full intent.

1. While establishing the parameters for performance pay, the House bill recognized that teachers at lower performing schools cannot compete fairly for increases. This proposed bill would provide a pay supplement to teachers at low performing schools for a year and continue only with improved performance of the school. So I understand that to be that at low performing schools, teacher pay is not tied to student performance in an assigned class, but the school-wide performance.
2. The bill seems to address merit pay for teachers who teach courses that have no student achievement data on which to base merit pay. Those teachers merit pay would be based on "student growth measures" and remain that way "an equally appropriate" measure is established by the school district. My read is that the districts are not forced to develop an end-of-course test for the P.E. teacher. "Student growth measures" is a new term for me and I will look into it, but I believe it refers to FCAT test results and the algorithm the DOE will develop to analyze results.

My sense was that this bill puts more emphasis on local control than SB736 and with that there are costs. My question remains - how much will this cost at the local level and where will the money come from?

http://www.myfloridahouse.gov/Sections/HouseNews/preview.aspx?PressReleaseId=176
http://www.myfloridahouse.gov/Sections/Committees/committeesdetail.aspx?SessionId=66&CommitteeId=2609

Missed a previous blog on merit pay in Florida? There are all here.
Avatar: www.clipartheaven.com

SB736: Entering the Twilight Zone

A fast moving thriller or spicy romance novel would be more entertaining and satisfying than reading a legislative analysis. Since accuracy and facts are hard to come by, I spent some time reading the now four analyses written for SB736. The analysts deserve credit for having written the document in clear English, free of mumbo jumbo, and easy to read. They cannot be blamed if the bill they describe does not make sense.

I noted a change in Section 5. Fiscal Impact Statement. In the two versions presented to the Appropriations Subcommittee, the sentence, the fiscal impact of this bill is indeterminate, has been deleted. Instead there is a paragraph describing what Race to the Top funding will cover and assistance to be provided by the DOE.

Florida’s Race to the Top (RTTT) grant will support the development of a revised teacher evaluation system as provided in this bill. Grant funds will enable the Department of Education to develop end-of-course assessments, item banks and components, such as the value-added model, for the evaluation system. The DOE will assist school districts in their development of assessment items that may be used for locally developed assessments.

During the next three years the grant will provide funding for the development of end-of-course exams in most subject areas. Additional resources may be necessary to maintain an assessment item bank or platform at the conclusion of the grant period.

District practices relating to the evaluation, compensation, and employment of instructional personnel and school administrators that are not consistent with the bill will need to be revised and implemented in accordance with bill implementation timelines.

SB736 is on the schedule for the Senate Budget committee on February 23. One can only hope that committee members are competent to conduct a complete cost analysis. Here are a few questions that need to be addressed:

1) Districts who agreed to participate in Race to the Top are recipients of funding. The analysis is silent on the costs required for those districts who chose not to participate and where the finds would come from.
2) While the DOE will provide "resources" to school districts, the analysis is silent on the amount of local monetary and manpower resources required to implement SB736 requirements. What is the fiscal impact on school districts and where will that funding come from?

SB736 is a complex bill with complex requirements. While the legislature and the Governor wrangle over further cuts to the education budget, the public has a right to have the facts on SB736.

Read the legislative analysis here: http://www.flsenate.gov/Session/Bill/2011/0736

Missed a blog on SB736 or want to read one again? You will find them all here.

SB736 Heads to Senate Education Appropriations Committee

Posted For

Sandra in Brevard





SB736 got a unanimous vote in the Senate Pre-K12 Education Committee and will now be considered by the Senate Budget Subcommittee on Education Pre-K12 Appropriation. The subcommittee is chaired by Senator David Simmons and Senator Bill Montford is VIce Chair. Committee members are Senators Nancy Detert, Paula Dockery, Anitere Flores, Evelyn Lynn, Jeremy Ring, Gary Siplin, and Stephen Wise. They will meet to discuss SB736 on February 15.

Some members of the Educations Appropriations subcommittee already showed some irritation with Governor Scott's budget plans for education that includes a 10 percent cut and a suggestion districts use this years funds to make up for the cut next year. Here are some highlights of their comments:

Sen. Evelyn Lynn, R-Ormond Beach, cast doubt on Scott's suggestion to plug some of the hole with stimulus money districts were given to spend for the current school year. "I just don't think that's as straight an arrow as I would expect," said Lynn, chairwoman of the Republican Senate Conference. "I look at it as a little smoke and mirrors."


"Regifting," Sen. Nancy Detert, R-Venice, interrupted.


"It seems a little bit improper," Lynn said.


Chairman David Simmons, R-Maitland, said some districts, including Broward - the state's second largest district, had already spent the money. Simmons said he's awaiting a report on all the districts.


"We'll have a better idea about whether this is real or not," Simmons said.


Senate budget chief J.D. Alexander pointed out to the governor's budget staff in his committee that their math did not add up correctly. While the staff showed cuts of $4.6 bilion in spending, Alexander pointed out the "real cut" adds up to less than $3 billion.

Reaction from the House side was similar:

“A 10 percent reduction is a significant cut,” said committee Chairwoman Marti Coley, R-Marianna.


Coley and Rep. Janet Adkins scolded Scott’s office for trying to "have it both ways" with the education budget. Scott said he’s against the use of federal stimulus money, but his office tacitly encourages school districts to use the money to boost per-pupil spending.


“It’s imperative that you go back and you redo the numbers,” said Adkins, R-Fernandina Beach.


"Committee members also questioned why budget categories had been renamed and changed. The so-called FEFP — the state's complicated, longtime school-funding formula — gets a new moniker, for example, and is now the Education Choice Fund.


Such changes make it hard to compare Scott's spending proposal with prior years' budgets, they said. "I don't know how the math adds up," said Rep. Kelli Stargel, R-Lakeland."

Reaction from Sandra In Brevard

Last year there was no detail on how much SB6 was going to cost. The Appropriations Committee must explain this year how much SB736 will cost and given the proposed cuts where the funds will come from precisely. And "fiscal impact is indeterminate" is not an acceptable response. If they do not develop a cost analysis, there's no point going forward.


Read more: http://www.miamiherald.com/2011/02/08/2057172/lawmakers-demand-budget-details.html#ixzz1Dgbq3R74

Read more: http://miamiherald.typepad.com/nakedpolitics/2011/02/gov-rick-scotts-k-12-budget-called-smoke-and-mirrors.html#ixzz1DgaK2EXH

http://blogs.orlandosentinel.com/news_politics/2011/02/scotts-4-6-billion-in-cuts-dont-add-up.html

http://www.orlandosentinel.com/news/politics/os-scott-education-budget-folo-20110208,0,5917927.story

To see comments on this article visit

http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/02/sb736-heads-to-senate-education.html

SB736: Fiscal Impact Indeterminate

Posted for

Sandra in Brevard




The Florida Senate Bill Analysis and Fiscal Impact Statement for SB736, filed by State Senator Stephen Wise, appeared on the PreK-12 Education Committee website. On page 11, Section V: Fiscal Impact Statement reads as follows:

A. Tax/Fees Issue: None

B. Private Sector Impact: None

C. Government Sector Impact:

"The fiscal impact of this bill is indeterminate.

According to the DOE, there will be additional costs to the districts for monitoring the use of evaluation criteria by supervisors and administrators.

As part of Florida's funding in Race to the Top, the DOE will assist school districts in their development of assessment items that may be used for locally developed assessments. Specifically, the DOE will provide the following:


Resources for districts to develop assessment items for "hard to measure" content areas, including Physical and Health Education, Fine Arts, and World Languages; Assessment items for core academic areas (Math, Social Studies,


Science, Language Arts, and Spanish) for grade levels and content areas that are not already tested by FCAT or state end-of-course assessments; and Development of a technology platform that will provide districts secure access to high-quality assessment items and tools for the creation and administration of student assessments.


The DOE notes that over the next three years the grant will provide funding for the development of end-of-course exams in most subject areas. The DOE also noted that additional resources or user charges will be necessary to maintain an assessment item bank or platform at the conclusion of the grant period.

According to the DOE, there are over 400 charter schools in Florida. The DOE reports that there will be a significant impact on its staff to review the evaluation systems for these schools.

It is not anticipated that the bill revises the total funds for instructional personnel and school administrator compensation."

Senator Wise seemed to indicate that he would focus on getting legislation written, but let the Senate Education Appropriations Committee figure out how to fund it. No cost analysis ever emerged for last year's effort (SB6).

Given the current state of the economy and Governor Scott's newly released budgetary measures, it is impossible to guess if SB736 is fundable even if a cost analysis emerges. With the proposed additional slashes to education funding, it would be unreasonable to divert a single remaining local dollar and/or resource to new tests and database development. Race to the Top funds extends to those school districts who signed on. The analysis does not address funding for districts that are not getting Race to the Top support.

Reports suggest that the Governor's proposal was not received with smiles and cheers in Tallahassee. While Scott proposes lowering the forced property tax, he cannot control local (county) education property taxes. If SB736 turns out to be an unfunded mandate, will local governments have to look at local increases they control?

The devil is in the details and we just don't have enough of that. Scott's proposal has to be voted on by the legislature and it looks like Scott needs to convince them. Simply stated, there must be no unfunded mandates and no encroachment on local control.

Read the full bill analysis here: http://www.flsenate.gov/Session/Bill/2011/736


To view comments on this article see

http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/02/sb736-fiscal-impact-indeterminate.html

Education Reform Like a Business: Funny Business Maybe?

NCR Pearson is the company that scores the FCAT results. When they bid for renewal of their contract, they underbid their competitor by $300 million, and won the $245 million dollar contract. Unfortunately, things didn't work out so well for Florida or for Pearson in 2010. In fact, things didn't work out so well in Wyoming, Minnesota, or Virginia either in returning results on time and generating some irregularities in scoring. Pearson paid Florida a fine of $15 million for the delay due to the "extraordinary difficulties in matching test results to each child's demographic information." They apologized saying that they had "underestimated the complexity of the work called for in the FCAT contract." Superintendents around the State challenged the results since their testing departments found drops in scores unusual. The State conducted two audits. The first was conducted by a company that was a sub-contractor to Pearson. The State then contracted a firm without any connection to Pearson, which found that the data was historically consistent with fluctuations in the past. Commissioner Smith then gave the FCAT results a "clean bill of health." What will this year bring?


Unsatisfied with the outcome, Alachua County school Superintendent Dan Boyd said all he can do is accept what the state's education commissioner has said regarding the audits.

"But there has been great consternation around the state with this, and we noticed some things we were concerned about with the scores, too," Boyd said.

There was particular concern when comparing student learning gains on this year's test with those of the prior year, especially for the lowest-performing students.

"And in looking at those scores, that was the problem with Pearson initially in matching those scores. So if they could not match them, how could they give us accurate results?" Boyd asked.

This year, Boyd is serving on a state-level FCAT review committee.

Florida has decided to develop end-of-course tests. Pearson was selected to develop these tests.


What business model is this? SIB to Captain Eagle, come in Captain, over......

http://www.orlandosentinel.com/news/education/os-fcat-test-company-problems-06-09-120100609,0,4407916.story

http://www.miamiherald.com/2010/06/10/1674316/fcat-test-producers-under-fire.html

http://www.gainesville.com/article/20100916/ARTICLES/100919558

http://www.gainesville.com/article/20100806/articles/8061007

To view reader comments on this article
http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/01/education-reform-like-business-funny.html

Merit Pay Bill: Senator Wise promises it won't be like last time

Posted for

Sandra in Brevard


We can only hope that Senator Wise is a man of his word. He says that his committee will devise a "thoughtful" bill. According to the Orlando Sentinel, Senator Wise will be holding a meeting this Friday, January 28, so that the "public can give testimony on the controversial topic." I don't know about any of Grumpyelder's readers, but I cannot travel to Tallahassee to give any testimony. These are the things I would like Senator Wise to be "thoughtful" about as he crafts the new bill:

It must be both fiscally possible and responsible given the budget deficit.

It should place no additional fiscal burdens on school districts nor on property owners.

It must recognize that there are differences in students. For example, exceptional ed teachers cannot be measured based on the performance of their students based on one test only. Merit pay has to be fair.

It must reasonable and allow the use of multiple measures to indicate student achievement. Tennessee has been doing just that for some time and recognized as a good practice.

It must recognize the long-standing concerns by parents and community members that schools have become testing mills, too much attention on tests and not enough on instruction. The bill should not increase the teaching to tests.

Anything else? What else should Senator Wise and the committee seriously consider???

http://www.orlandosentinel.com/news/education/os-florida-merit-pay-superintendents-20110126,0,6284635.story

To view reader comments on this article, see

http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/01/merit-pay-bill-senator-wise-promises-it.html

Education Reform: If the roof leaks, call 911

Posted for

  Sandra in Brevard




For at least the last 12 years, we have heard again and again that schools, students, and teachers are failing. For at least the last 12 years, national and state initiatives have centered on fixing that problem through accountability and testing initiatives. No Child Left Behind (NCLB) poured billions into the effort while schools struggled to meet the implementation requirements. Race to the Top is more of the same. For the same period of time, Florida poured millions into the FCAT, grading schools, and now it is on its way out to be replaced by another generation of tests.
What business would survive if after more than a decade, there was no return on investment? What business would pour money year after year in fixing something with the same tools and year after year see no progress? What business would fail to go back and examine the problem they were trying to solve?

Maybe the conventional wisdom "if ain't broke, don't fix it" should be reconsidered in educational reform.

University of Florida researchers ‘’borrowed ‘lifestyle segmentation' profiling methods used by direct marketers and political strategists to classify every student into one of several lifestyle groups (four in Bay County, three in Alachua), each based on a common set of values, income level, spending patterns, education level, ethnic diversity of neighborhood and other shared traits." The researchers used this data to examine the relationship between each group’s lifestyle profile and their math and reading scores on the Florida Comprehensive Assessment Test, the state’s standardized exam used to evaluate student and school performance. Researcher. The results indicated that "the most affluent lifestyle group registered the highest FCAT scores, the second richest group ranked second in test scores, and so on. On the math tests, the gap between the highest and lowest scoring lifestyle groups was more than two grade levels." The lead investigator, UF Professor Harry Daniels, said: “The testing patterns in both counties virtually mirrored each other. Every lifestyle group improved in FCAT scores from year to year until the 10th grade exam (which students must pass to graduate high school), when improvement leveled off. But they all improved at the same rate, so the achievement gap persisted year to year.”

Instead of continuing a path of more of the same, perhaps real reform comes in the form of a different set of educational programs. Perhaps it would be a better idea to spend money on vocational programs. Looking back historically, good jobs get poor families out of poverty and often are in trades. Even in this economy, we still need electricians, auto mechanics, and a variety of positions in the health care field that require A.A. or A.A.S. degrees. These jobs require solid math and literacy skills that high school vocational programs can develop. This doesn't restrict any socio-economic group from pursuing a university directed education. However, since the FCAT is on its last legs for high school graduation to be replaced by end-of-course exams, data results might change if students had different choices.

The Eli and Edythe Broad Foundation published a report on the “crisis” in US education and convened 30 individuals to make recommendations on how to fix the problem. This “crisis” is based upon the performance of US students on the Programme of International Student Assessment (PISA). A cross section of students at public and private schools, between the ages of 15 years 3 months old and 16 years 2 months, are selected from schools that voluntarily participate. A minimum of 4,500 per country are required to participate in the test. Shanghai placed #1 on test results, but Shanghai is not a country. Approximately 35% of Chinese students do not make it to high school. Singapore is in #2 position. With a population of 4,424,133, the central government controls and manages the country’s school system, which based on what I can locate, includes technical and vocational training schools for high school students. The language of instruction in Singapore is English. The United States has a population of 308,400,408 and individual states control educational standards and testing initiatives. The FCAT is an example of a state-centric exam.

Conclusion

Today, I have no idea what the true condition of US education is. I do not believe we are in a “crisis.” There is no data to support that. Performance on an international test is insufficient to make such a claim. We do know that US students drop out at unacceptable levels. We do know that income levels have something to do with student achievement on tests. The solutions to gather data and produce more tests are a continuation of more of the same “solutions”. The Florida legislature has done an incomplete analysis of the problem and that is where the failure is.


More information on the UF study can be found here:
 http://news.ufl.edu/2010/03/22/school-success/

More information on PISA can be found here:
http://www.pisa.oecd.org/pages/0,3417,en_32252351_32235731_1_1_1_1_1,00.html


© SandraInBrevard, All Rights Reserved

To see comments on this article visit

http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/01/education-reform-if-roof-leaks-call-911.html

Follow Up to Data Mining

Posted for

Sandra in Brevard





The United Way is hosting community conversations to promote civil discourse on education reform across Florida, North Carolina, and Tennessee this month. This “listening tour” on teacher effectiveness is funded by the Bill and Melinda Gates Foundation. If the scheduled ones are inconvenient, the United Way offers a way to form a convenient one, call them and they will explain how. Looks to me as though a civil conversation on educational reform is happening right here at Grumpyelder's place where there is no need to leave any personal information or details that could be mined later by an algorithm.

http://wiki.tothevillagesquare.org/display/EET/About+the+grant

Here’s some additional information:

Hillsborough County has a grant from the Bill and Melinda Gates Foundation.

According to the Broad Foundation, Data Quality Control (DQC) begia assisting states in 2005 to build educational longitudinal data bases. The DQC website, The Bill and Melinda Gates Foundation is listed as its founder.

The Eli and Edythe Broad Foundation is a national venture philanthropy established by philanthropist Eli Broad to advance entrepreneurship for the public good in education, science and the arts. The Foundation funds the Broad Residency, which searches for individuals with MBA's and in industry for candidates to take rapid training to take on positions as superintendents and other managerial positions in our nation's school districts. The Broad Foundation subsidizes salaries once hired on. The Broad Center announced it's placed "the largest class of 42 early career executives into 28 public education systems, expanding for the first tie into state departments of education." One Broad Resident now works for Hillsborough County Public Schools.

The Broad Foundation received a $3.6 million grant from the Bill and Melinda Gates Foundation to recruit and train as many as 18 Broad Residents over the next four years to provide management support to school districts and charter management organizations addressing the issue of teacher effectiveness. "Broad Residents will help school systems dramatically improve the recruitment, selection, training, placement and evaluation of teachers". The Gates Foundation grant is the first multi-million-dollar grant The Broad Residency has received from a funder other than The Eli and Edythe Broad Foundation.

"http://www.broadeducation.org/asset/0-100908tbrnewclassgates.pdf

All Broad Residents have M.B.A.s or other advanced degrees. Seventy-four percent of this year’s class, selected from a candidate pool of more than 2,500 applicants, come from leading business and law schools such as Harvard University, Duke University or the University of Michigan. Participants have an average of 10 years of experience, typically from a Fortune 500 or other major company. Fifty-two percent are people of color. The Broad Residency continues to be far more selective—at 2 percent—than the highest-rated M.B.A. programs. The Broad Residency (www.broadresidency.org) pays 50 percent of each Resident’s salary the first year, and 25 percent the second year, with the partner organization paying the balance, except where a Resident is already employed by that organization.

http://www.broadeducation.org/asset/0-100908tbrnewclassgates.pdf



© SandraInBrevard, All Rights Reserved

To see comments on this article,visit

http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/01/follow-up-to-data-mining.html

Data Mining: An Education Reform Strategy

Posted for Sandra in BrevardIf you got mad a few years ago, when you found out Motor Vehicles was sellng information, this won't make you at all happy


If you buy a book through Amazon, rent a movie through Netflix, or have a Facebook account, your information and choices are "mined" to market new products catered to what the data reveals about you. In these large databases, your choices are compared with others and a book you liked might be offered to others who seem to have similar tastes or interests. Specialized algorithms are developed that "mine" in an effective process to sell products.

Wikipedia defines algorithms in part this way:
"Algorithms are essential to the way computers process information. Many computer programs contain algorithms that specify the specific instructions a computer should perform (in a specific order) to carry out a specified task, such as calculating employees' paychecks or printing students' report cards."

NY Times contributor Seth Freeman wrote a clever article this week titled "Me and My Algorithm" of which he said:

“If this is a case of my algorithm, my cyber personal shopper, coach, guardian angel and avatar, knowing me better than I know myself, I really do need to figure out why I, a guy, get repeated offers — tied to a e-mails on vastly different subjects — for mastectomy bras and for something called a vaginal ring. Is the idea that these items make lovely gifts? Since articles I have written have circulated through the Internet by e-mail, it could easily turn out that my algorithm will soon get the opportunity to read what I have had to say about it here. What, I wonder, will it think?” (1)
 
Last year, Bill Gates and other Microsoft executives obtained a patent for a personal data mining system that "would analyze information and make recommendations with the goal of aiding a person's decisions and improving quality of life. The patent abstract described the system this way: "Personal data mining mechanisms and methods are employed to identify relevant information that otherwise would likely remain undiscovered." Users supply personal data that can be analyzed in conjunction with data associated with a plurality of other users to provide useful information that can improve business operations and/or quality of life. Personal data can be mined alone or in conjunction with third party data to identify correlations amongst the data and associated users. Applications or services can interact with such data and present it to users in a myriad of manners, for instance as notifications of opportunities. Of course, it's not all about improving lives: Further down, the patent explains that "such data can be afforded to businesses involved in market analysis, or the like, in a manner that balances privacy issues of users with demand for high quality information from businesses." (2)
 
Building Longitudinal Data Systems for Education

What does this have to do with education? Plenty. There is a widespread belief that the development of longitudinal data, from early childhood through the 12th grade and beyond is a necessary element to educational reform. The Data Quality Campaign (DQC), "a national, collaborative effort to encourage and support state policymakers to improve the availability and use of high-quality education data to improve student achievement." The organization articulates a widespread belief that "States have made remarkable progress in developing longitudinal data systems that can follow student progress over time, from early childhood through 12th grade and into postsecondary education through implementation of the 10 Essential Elements. The 10 State Actions are the fundamental steps states must put in place to change the culture around how data are used to inform decisions to improve system and student performance."

Florida received a federal grant for $9,975,288 with funding starting in July 2010 and ending in June 2013 and cited these major outcomes in their proposal
 

a) Upgrade the four major source data systems that are incorporated into Florida’s Education Data Warehouse (EDW)

b) Employ a unique identifier system so that social security numbers are no longer the key field for tracking students between the Local Education Agencies and the State

c) Provide several different reporting capabilities for use by a myriad of stakeholders

d) Implement a data mining tool for FLDOE to analyze and evaluate its program and policies more efficiently and effectively (3)

The Data Quality Campaign reaffirmed that "Florida is among the top states in collecting data (10 of 10 criteria along with 11 other states) and using it (5 of 10 criteria, better than all but two states). "When states collect the most relevant data and are able to match individual student records over time, they can answer the questions that are at the core of educational effectiveness." (4) According to their website, the founding father of the Data Quality Campaign is the Bill & Melinda Gates Foundation, with additional support from Casey Family Programs, Lumina Foundation for Education, Michael & Susan Dell Foundation, and the Pew Charitable Trusts. You can follow DQC on Facebook.

DQC’s executive director believes that there is education data collected that is not necessary and cited Kansas and Tennessee as “leaders in establishing rules for data control.” However, the Fordham Law School Center on Law and Information Policy conducted a study (5) on the massive data collection efforts and concluded that states "are collecting far more information than necessary, failing to take appropriate measures to safeguard student privacy and protect them from data misuse, and failing to comply with the Family Educational Rights and Privacy Act (FERPA). Fordham's investigation also reveals that 80% of the states "do not have a system to delete student records. "Fordham law professor Joel R. Reidenberg, who oversaw this study, had this to say of the Center’s findings:

“Ten, 15 years later, these kids are adults, and information from their elementary, middle and high school years will easily be exposed by hackers and others who put it to misuse. States, he said, "are trampling the privacy interests of those students." (6)
 
Conclusion

Bill Gates and the entire computer industry need a literate population with financial means to buy and make use of their products. Therefore, at some level, these efforts are intended to spur improvements. I do not mean to suggest any nefarious intent. Clearly, there is business development intent. Then, I wondered what other benefits a massive data collection has. Could it be a way for an industry, like the computer industry, to be able to identify minds early on with potential to join that workforce, nurture them, and ensure that the U.S. has sufficient minds here versus importing from abroad. Right now, the jobs in this sector are blooming in China and India and likely a destination for unemployed U.S. computer guru's, leaving the potential for a U.S. brain drain. Whatever the reasons, I find it troubling. What do you think?
 

1.     http://www.nytimes.com/2011/01/18/opinion/18freeman.html?_r=2
2.    http://www.techflash.com/seattle/2010/02/gates_ozzie_other_microsoft_execs_patent_personal_data_mining.html
3     http://nces.ed.gov/programs/slds/state.asp?stateabbr=FL
4.     http://www.tampabay.com/blogs/gradebook/2010/01/florida-among-top-states-in-education-data-collection-and-use.html
5.     http://law.fordham.edu/center-on-law-and-information-policy/14769.htm
6.     http://www.washingtonpost.com/wp-dyn/content/article/2009/10/27/AR2009102703562.html

© SandraInBrevard, All Rights Reserved

To see comments on this article visit

http://grumpyelder-todayimgrumpyabout.blogspot.com/2011/01/data-mining-education-reform-strategy.html