Make Decisions like a Superforecaster

Superforecasting: The Art and Science of Prediction by [Philip E. Tetlock, Dan Gardner]

Was Osama bin Laden in that compound in Abbottabad, Pakistan?  Leon Panetta sought answers from each of the analysts that were charged with providing direction to Panetta and eventually President Obama.  The collective wisdom  of these staffers eventually persuaded President Obama to approve military action that would indeed take out one of the masterminds of the 9-11 attacks.
While we may not be making life or death decisions of this magnitude, the lessons learned about decision making from Phillip Tetlock and Dan Gardner’s book, Superforecasting The Art and Science of Predicting, contain hard earned wisdom that can apply to a various contexts. Effect decision makers can benefit from the rigorous analysis and protocols that lead to the best decisions.  Here are three takeaways for decision making.
  • Expertise in any area is developed through careful study, trial and error, feedback, and determination.
  • Teams can make great decisions only when there is healthy conflict and skepticism.
  • So long as the overarching goal is clear, decisions should alway be made closest to the point of the action.


Whatever goal one finds worth pursuing it is rarely achieved through brute strength or pure intellect.  Raw materials do matter but they can only take one so far in developing a talent or skill. The superforecasters highlighted in this book had similar characteristics.  They exposed their judgments to analysis and feedback and made adjustments based on that feedback. This required humility and grit to accept errors and rethink their course of action.  They were also not easily fooled by positive results as sometimes their prediction were correct only by luck.  They scrutinized their process as rigorously as their outcomes to unearth flaws in thinking that could be detrimental if left unattended. Their expertise was earned through skeptical self reflection and interminable hypothesis testing.
“The strongest predictor of rising into the ranks of superforecasters is perpetual beta, the degree to which one is committed to belief updating and self-improvement.”
– Phillip Tetlock, Dan Gardner from Superforecasting


Which is more important, the members of your team or the processes you use to make decisions?   Let’s take John F. Kennedy’s dream team, known as Camelot.  One of their first tests was the Bay of Pigs invasion, a total fiasco that led to the loss of Cuban and American lives and the solidifying of the Castro regime and Cuban/American animosity that remains with us 70 years later.  When this plan failed miserably, did Kennedy replace his staff? No!  Instead, they changed they way they made decisions and fostered skepticism (to avoid groupthink) and Kennedy promoted debate, constructive confrontation, and withheld his opinion until all sides were debated and challenged. This second approach led to much more intense and stressful meeting dynamics, nevertheless the results was more nuanced and mature decision making that, in part, led to the successful avoidance of nuclear war and disaster during the Cuban Missile Crisis a couple years later.  Teams that learn to challenge, probe, question, and analyze points of view produce better decisions.


Otherwise known as local control. Tetlock and Gardner share the philosophy, especially in military leaders, to privilege information on the ground.  Clearly articulated overarching goals are best achieved when those who are taking action have the freedom to make decisions based on their local context.  The takeaway for leaders is to be clear about your vision and mission and flexible about how your teams accomplish those goals.  Autonomy is one of the key motivators that drives all of us to perform.
Therefore, leaders can create the conditions for effective decisions by subjecting all processes and outcomes to scrutiny and analysis, building teams that discuss all sides with candor, and relying on implementers to determine the best course of action. All of this begins with a healthy does of self reflection and humility.

What you See is What You’re Looking For

I’ve got a backlog of books that I’ve read recently and this one stands out because of its excellent writing, thorough research, and honest appraisal of the Success Academy school network of charter schools.

How the Other Half Learns by Robert Pondiscio

This book will more than likely not change your mind about charter schools and school choice.  Mr. Pondiscio likes to use the phrase “Rohrshack test” when discussing the strengths and weaknesses of Success Academy in New York City.  If you are prone to love charter schools then you will find much to like about this school that gets better results than any school in New York City.  If you are prone to find fault with privatized education and schools that emphasizes strict discipline, then you will find plenty to criticize about this network of schools.

The beauty of this book is that Robert Pondiscio spent an entire school year with nearly unfettered access to leadership, staff, students, and families.  He tells a nuanced story from the perspective of those who are most directly impacted by a school and it left me moved by the passion and commitment of the educators and the determination of parents to give their children the best possible chance for success.  There are some questionable practices that he uncovers, but there is no question that the schools that Eva Moskowitz has built are producing exceptional results and partnering with willing families who only want the best for their kids.  This quote from Principal Elizabeth Vandlik to her staff at the start of the school year epitomizes the ethos of Success Academy:

The kids are the wonderful, beautiful work that we do.  We’re the ones who need to grow and change.



Chicken or the Egg

Extended Quote from Paul Bambrick-Santoyo in Leverage Leadership 2.0

One of the great debates of school leadership is what should take priority: student instruction or student culture.  In the culture camp some argue that without order, joy, and respect, academic success is impossible.  In their eyes, the game plan should be to “delay” instruction until culture is “right.”  On the other side, some argue that instruction creates culture, and that as teachers create engaging and rigorous lessons, student conduct and attitudes will naturally improve. Both views are badly flawed.  If instruction is strong but culture is weak, a school’s success is crippled: newer teachers face serious discipline challenges, students experience radical inconsistency between classes, and core values cannot be taught.  Yet at schools that decide to “wait” on improving instruction, the end result is often order without rigor, a “false positive” that looks like education but is anything but.  The truth is that both instruction and culture are vital, and both must be led simultaneously.  Without this, neither can succeed. (p. 7-8)


Fullan on finding and Implementing Good Ideas

I argue in this book that most good ideas come from first examining good practices of others, especially practices that are getting results in difficult circumstances. The second step is to try out the new ideas yourself. The third involved drawing conclusions from what you have learned, and then expanding on those conclusions. Deliberative doing is the core learning method for effective learners

Michael Fullan, Change Leader

What Fullan describes here is the concept of Positive Deviance. This is the idea that the best way to make improvements in any system is to study and apply the practices of the bright spots and successes. Google has made a science of studying their managers to uncover the different behaviors that produce the best and worst results. They then share and encourage implementation of those practices. This relentless focus on uncovering and highlighting these great ideas is the heart of Fullan’s strategy of the Change Leader.

Fullan on Motivation Through Passion or Accomplishment

It is being in the moment of a successful endeavor that fuels passion, not the dreaming of it. Thus, exhorting people to have greater moral commitment is often less effective than helping them get new experiences that activate their moral purpose. The establishment of new practices and experiences galvanizes passion. This is the essence of the change leader: the capacity to generate energy and passion in others through action.

Michael Fullan, Change Leader

In the opening chapter of Change Leader, Fullan continuously emphasizes the primacy if action over theory. Practice is the driver and learning is the outcome.

Silos vs. Networks

Last week I had the opportunity to visit Long Beach Unified School District with the purpose of learning about their  Linked Learning and College and Career pathways.  Their programs are truly inspirational and encouraging.  We heard from their Superintendent Chris Steinhauser and several members of his staff who described the history and nature of their work to develop career pathways in all of their high schools.

One of the statements that was repeated over and over again in their explanation was their insistence in not working in silos.  They were intentional about sharing information throughout their organization in order to develop plans that integrated the best thinking of everyone in their school community.  Their discussion about silos reminded me of Hugh Howey’s trilogy (Wool, Shift, and Dust) that describes a post apocalyptic world in which society is organized into individual silos (none of which has knowledge of the existence of the other).

Wool Omnibus Edition

This dystopian novel illustrates the many negatives for developing around the concept of silos, which traditionally, has been the organizing premise of most education institutions.  Why should we move beyond silos to a more networked and connected systems?

Silos do provide some superficial benefits.


Silos are organized

Silos are compact

Silos are manageable

Silos are comfortable (to an extent)

Silos produce predictability


And, how do silos accomplish these aims?


Silos thrive on secrecy

Silos thrive on darkness

Silos thrive on control

Silos thrive on specialization and separation


Networks on the other hand may appear to be a downgrade.


Networks are decentralized

Networks are messy

Networks are complicated

Networks lack control

Networks produce ambiguity


And what are the ingredients that produce healthy and growing networks?


Networks flourish in the light

Networks flourish with transparency

Networks flourish with sharing

Networks flourish with interdependence


The end result, as Long Beach Unified has demonstrated so clearly, is that improvement and innovation are much more likely to be produced in networks of connected educators than in silos of independent operators.  What will we do today to move from silos to networks?







Pressfield and Coyne on Sharing Your Gift

It may help to think of it this way. If you were meant to cure cancer or write a symphony or crack cold fusion and you don’t do it, you not only hurt yourself, even destroy yourself.  You hurt your children.  You hurt me. You hurt the planet.

You shame the angels who watch over you and you spite the Almighty, who created you and only you with your unique gifts, for the sole purpose of nudging the human race one millimeter farther along its path back to God.

Creative work is not a selfish act or a bid for attention on the part of the actor  It’s a gift to the world and every being in it.  Don’t cheat us of your contribution. give us what you’ve got.

Steven Pressfield and Shawn Coyne, The Art of War

Pressfield and Coyne on Professionalism

This quote made me think of the perfect response by the University of Virginia coach (having just been the first #1 seed of the NCAA tournament to lose to a #16 seed) who spoke eloquently about the preference of being in the arena and the risk all face when stepping into that action.

The professional keeps his eye on the doughnut and not on the hole.  He reminds himself it’s better to be in the arena, getting stomped by the bull, than to be up in the stands or out in the parking lot.

Steven Pressfield and Shawyn Coyne, The War of Art

Datawise: Male Dropout Rate Raises Many Important Questions

The power of a well designed data visualization is the quality of the questions that it raises.  This simple graph of the dropout rate since 2000 shows the gap between the dropout rate of black and white male students.  Statements by the author explore the concept of black teachers’ impact on black students’ dropout risk and college aspirations.

Link to Tableau Public

Gloria Ladson-Billings has a different take on the same topic advocating for the need for white students to have black teachers.  She describes the reaction of many of her university students to the first black teacher they had ever encountered.

They seemed amazed that I had both a wide and deep knowledge of a variety of subject areas and knew how to encourage and draw more out of them than they thought possible. My hope is that their experience with me makes them walk into classrooms filled with Black children and say, “there could be doctors, lawyers, engineers, professors, inventors, and teachers in here,” rather than assume that their black skins limited their intellectual possibilities.

Gloria Ladson-Billings

Students can learn from teachers who look like them.

Students can learn from teachers who look nothing like them.

Any way that we can increase the diversity of the teaching force will positively impact every student in the school.  All of our students will benefit from that increased diversity and students who have historically been underrepresented will benefit even more.



The Wisdom of Whole School, Ongoing, Long-View, Always Learning, Never Ending, School Reform

Photo by jesse orrico on Unsplash

But we love the overnight success story!

It’s so alluring.

It grabs the headlines.

And, it’s so wrong.

Lasting change rarely comes quickly.  The need to build a strong foundation should outweigh our appetite for immediate results.

John Wooden, who I have written about here, won 10 national championships at UCLA and yet, it took him 16 years to win his first.

We sometimes substitute a commitment to learn about our organization and the people with in it for a sense of urgency and quick fixes.  Short term results not only are difficult to sustain, but almost always come with unintended negative consequences.

We need to take the long view in school reform.  A proper foundation begins with instilling a professional learning culture.   Short cuts such as canned programs may provide an initial boost, but it can keep us from doing the harder work of questioning our practices and building new models of teaching and learning together.

One of psychologist Jordan Peterson’s 12 Rules for Life is,  “Don’t do what is expedient, do what is necessary.”

That should be our motto as we look towards a future of improved learning opportunities, we will need to make difficult decisions and engage in the hard work of improving practice one classroom and one staff room at a time.

So, let’s make our schools a little better today.  Let’s focus on what is necessary, not expedient to improve the culture for teaching and learning.