Skip to content
February 17, 2012 / B.J.D.Armas

Program Evaluation Ch. 1 Royse

Motivations for Program Evaluation

We want to show:

  1. that clients are being helped
  2. that clients are satisfied with our services
  3. that the program has an impact on some social problem
  4. that a program has worth
  5. that one program or approach is better than another
  6. that the program needs additional staff or resources
  7. that staff are well utilized

We want to know:

  1. Are clients being helped?
  2. Are clients satisfied with the services received?
  3. Has the program made any real difference?
  4. Does the program deserve the amount of money spent on it?
  5. Is the new intervention better than the old?
  6. How do we improve this program?
  7. Do staff make efficient use of their time?
February 13, 2012 / B.J.D.Armas

Qualitative Methods in Evaluation

  • Qualitative Topical Interest Group of the American Evaluation Association.
  • Ethnography, grounded theory, narrative analysis.
  • Empirical and systematic.
  • Goals of evaluation:
  • How and why of the program.  Suitable for formative evaluation.
  • Important for staff morale, executive deciion making, misunderstandings, and client perceptions.
  • Has immediate benefits for participants.
  • Cannot report on program outcomes with precision.
  • But vivid descriptions of how boys respond to program.
  • Qual useful in formative phases —- identifying ways to shape a class to overcome resistance.
  • Qual useful in process evaluation to examine how well program instructors were communicating a message or doing homework
  • Units of analysis = individuals:  also could be agencies, group homes, or settings that are arenas of human activity organized around a program.
  • Focus group interviewing and rapid ethnographic assessment:
  • Action Evaluation:  Commitment to social justice
  • Obstacles to praxis-oriented approach that promotes social justice and empowerment:  1)  Spoonsoring institutions hate change 2)  Projects are bogged down in differences in goals.  3)  Professional evaluators find themselves in difficult position if contributing to the formation of a new oppressive elite.
  • Easing entry = key informants.
  • Payback = sharing final results, offering on-site feedback
  • Deviant case sampling = learn about outliers.  vs. typical case sampling.
  • Maximum variation sampling = seek representativeness — cases that cut across variations in processes.
  • Snowball sampling = isolated, hard-to reach or suspicious of outsiders.
  • Convenience sampling = taking advantage of cases at hand
  • Data Collection
  • On-site observation – field notes, participant observers.  Aim for unbiased observation of physical setting — spatial layout, visual decor — draw floor plan and layout of the agency.
  • Observations:  have to zero in on activity
  • Reflexivity
  • Interview Guides:  attitudes = opinions, feelings/emotions = emotional reactions, knowledge = test of how much he knows about certain facts or verifiable information, and behaviors = actions taken.
  • Focus group = no more than 10, lasting an hour or more.  Convened by facilitator who leads group thru series of questions designed to make participants feel at ease.
  • Documents are essential source of data (Patton 1987) recommends early request of documents
  • Saturation occurs when data analyses reveal repetition and redundancy, new data confirms existing findings rather than expand upon them
  • Data management = find meaning units in data and develop a conceptual scheme.
  • Codes
  • Use case studies in final reports
  • Synchronic and diachronic analysis.  Etic does = outsider interpretations
  • Qual researchers use credibiity and trustworthiness vs. reliability and validity
  • Leaving an audit trail = document steps taken and decisions made during analyses
February 13, 2012 / B.J.D.Armas

Rubbish Theory


  • Transient (Value Decreasing) -> Rubbish (No Value) -> Durable (Value Increasing)
  • hose who do the best fieldwork merge with their subjects must at the same time sever their links with their discipline
  • Thing doesn’t just die….continues to exist in timeless and valueless limbo where it has the chance of being discovered
  • Phlegm is rubbish, but recently had a noble connotation;  English phlegm was expectorated from splendidly stiff upper lips.  Sweat also had good, honest, noble connotation.
  • Communicable objects assigned to three categories:  transient, rubbish, and durable
  • Increasing economic vlue, once they have entered the durable category, is accompanied by increasing aesthetic value
  • 2 Tribes:  Knockers-Through and Ron-and-Cliffs:  Were knocking-through, were the first people to come and live here.
  • Knocker-Through wants to get house to durability, Ron-and-Cliff has no access to durability
  • Historians rearrange ancestors for political ends;   as it crystallizes:  telescoping = structural amnesia
  • The fact that objects must be processed in a systematic ways means that they must become ordered in some way.
  • Rubbish is a universal feature of socio-cultrual systems;  its is a consequence of the impossibility of ever fully specifying the relationship b/w a process and its cognition from a position which must always be contained by such a relationship
  • We only notice rubbish when it is in the wrong place
  • That which is discarded but not visible, because it does not intrude, is not a cultural categroy at all, it is simply residual to the entire category system
  • We can individually see contents of dustbins and handkerchiefs, yet collectively we suppress the sight
  • Gypsies provoke response from sedentary residents
  • Homelessness arises when boundaries are drawn so clearl and so all-embracingly that they are of the p and not-p type, and their control extends so absolutely and so far that that which happens to be neither p nor not-p can find a squalid over-looked corner to make its own.
  • We all recognize rubbish;  no questions of degree
  • Dogs are dogs = self-evident = certainty
  • Science outrages value
  • [For scientists] Truth was much less ethical, much less tidy, and much more human.  All that happened was that those who subscribed to the old theory eventually died.  Once this is accepted, scientific progress becomes more like scientific meandering, and the idea that a science has only reached maturity when it can afford to ignore its history has to be stood on its head.  Only by suppressing its history can a science present the appearance of maturity.
  • The rate at which items enter durable category = greater than rate at which they are removed from circulation
  • Art is always a high status activity since artists handle the symbols of society…politicians and engineers handle symbols only incidentally and implicitly….artists deal entirely and explicity in symbols…Art = creation of durability
  • Monster exclusion
  • Transaction = People acquire through involvement with people and objects
  • People do not just interact:  they interact over something.
  • We do not have access to raw nature but only its socially processed form
  • Creativity = that which alters world view
  • Two types of creativity:  1)  alters internal arrangement, but not content of world view domain  2)  alters content of the domain
  • Four kinds of creativity:  entrepreeurial, wilderness, negative, serendipitous
  • Knowledge = side-effect of society -> asymptotic approach to something called truth
November 15, 2011 / B.J.D.Armas

Garbage Land

  • Trash can does not exist until municipal collections organized
  • Stove = principal means of dispoal.
  • Food scraps go to farm animals
  • Consumer goods rare and expensive
  • Tin cans saved for storage or scoops, jars preserve food
  • Old clothes repaired, made into new clothes, used for quilting, mattress stuffing, rags, or rags
  • In days of household economy, castoffs and scraps are used to barter
  • Composters = treasure trove of nitrogen
  • Litter = tool of anarchy
  • Scavenging pigs and dogs consume waste
  • Fresh kills opens in 1945
  • History of garbage in NY = history of interim solutions
  • In the neighborhood where I live, the garbage is boxed and gift-wrapped
  • Sanitation garage = Brimming with stuff
  • Transferring objects = lighter and cleaner
  • Money to be made from burying trash
  • 1998 NY State Department of Health study finds escaping landfill gases contribute to fourfold inrease in bladder cancer and leukemia rates in women who live within 250 feet of thirty-eight upstate landfills
  • Incinerators competes with attempts to reduce volume of waste…if you pay to incinerate, why bother to reduce, reuse, and recycle
  • Underworld…Jesse Detwiler teaches UCLA students that garbage has its own momentum, that it has the power to shape people.  Pushed to edges, garbage also pushed back.  People were compelled t develop an organized response.  This meant that they had to come up with a resourceful means of disposal and build a social structure to carry it out—worker, managers, haulers, scavengers.  In Detwiler’s view, garbage acts as evolutionary force, helping to shape early man.  It forces us to develop the logic and rigor that would lead to systematic investigations of reality, to science, art, music, mathematics.
  • Dump = negative space , but could be filled and capped and something alive and new could be created in its place, landfill has the potential to be an amenity
  • Most landiflls = muffiers than composters
  • Raw landfill gas contains carcinogenic air pollutants, but burning it in a flare, an engine or turbine reduces toxicity
  • Solution to pollution is dilution – enviro engineers
  • The worst things we threw out were the things that had once been alive;  organics render everything in the can loathsome to touch and smell:  carbon-based stuff that could be transformed into valuable horticultural products;  food waste makes trash heavy and wet
  • Biocycle = state of garbage in America report
  • According to NY Department of Sanitation, edible and inedible food debris = 15% of household garbage within the city
  • LA population of 3.7 million, mixes quarter of 300,000 tons of yard trimmings, more than 30% of residential waste stream with sludge from wastewater treatment plant to make fertilizer called TOPGRO
  • Smell of garbage is sulfur and nitrogenous compounds.  ‘as waste heats up, chemical compounds are liberated by microbes, but the first ones we notice are sulferous.  With freezing, the gas molecules that carry order become unavailable.  It’s called partitioning, when a chemical changes from one form to another.  Over time, and with termperature, the smell becomes more available to the nose.
  • Rotting flesh of animals produces organic acids that attract flies, which aim to deposit their eggs ina  food source that will nourish their growing larvage.
  • At room temperature, you could smell acetone.
  • Putrid odors are potent in making people want to flee in disgust — also causes increased heart rate and shallow breathing and can lead to nausea
  • Decomposition slows dramatically in the winter
  • Attending garbage events around the city – roundtable on recycling, city council meetings, garbage art shows, garbage movies
  • Anaerobic food digesters – builds digester to handle pre- and postconsumer food waste from commercial establishments and homes
  • Worms = tiny mouths, and slow and fussy easters…won’t eat onions, orange rinds, driedpout lemons or banana peels…can’t withstand the cold
  • In times of abundance, no one cares about conservation
  • Food waste disposers invented in 1935
  • Remaking the Way we Make Things:  3 Categories  a)  Consumables, biodegradable stuff = shampoo bottles made of beets, fabrics free of toxins, mutagens, and endocrine disruptors,  b)  Durables like TVs and cars, returned to manufacturers as tech nutrients and used as food in manufacturing systems.  c)  unmarketables like nuclear waste, dioxin, and chromium-tanned leather would not longer be produced or sold
  • George Waring finds that transforming NYC’s organic waste to fertilizer and grease is easy
  • Wastepaper = American’s largest export to other countries
  • China imports more than $1 billion of American scrap
  • Beer and soda == most recycled consumer product
  • Electronic waste = 40% of lead in municipal dumps
  • Takes 1.8 tons of raw material to manufacture desktop pc and monitor
  • In the garbage world, weight = money
  • Keep America Beautiful = consumer is responsible
  • That which we can’t afford to process is sent overseas to China
  • *Technology does the organizing of these wastes
  • Paper and packaging = 35-40% of household waste in North America
  • For every 100 pounds or product, 3200 lbs of waste are generated – Paul Hawken
  • Recycling = wise, because it’s far-sighted
  • To shop = American
  • More wealth = more waste
  • Low-flow showerheads, energy star refrigerators, hybrid cars, converting to biodiesel
  • What we discard makes us rich
  • *Hoarding = no social or cultural capital to organize
  • Urban Ore = Berkeley
  • Municipal solid waste = 2% of nation’s waste, with remainder nonhazardous industrial waste, mining, agricultural, and hazardous waste
  • Consumer goods has to be less toxic, designed for recycling and returned to their makers
  • Burying or burning waste spurs more resource extraction to make more products
  • Plastics = number 1 category of litter to collect
  • Can’t keep burying waste, never escape our own mess
  • If we don’t wake up and make the connection between our economy and the environment, the planet will eventually do it for us.  And it won’t be pretty.
October 20, 2011 / B.J.D.Armas

Mutual Focus/Entrainment

  1. Two or more pple physically assembled in same place
  2. Boundaries to outsiders
  3. Focus attention upon common object or activity
  4. Share common mood or emotional experience

4 Outcomes of interaction rituals

  1. Group solidarity, feeling of membership
  2. Emotionalenergy in the individual – feeling of conifdence, elation, strength, enthuiasm, initiative in taking action
  3. Symbols that represnt the group – emblems pple feel ar part of themselves, treating symbols with respect and defend against disrespect of outsiders or renegade insiders
  4. Feelings of morality – sense of rightness adhering to the group

Forced rituals = draining

Criteria of ritual success or failure:

  • Empty ritual
  • Merely ceremonial
  • fell flat
  • Energy-draining effects after job interviews, academic job talks = interaction fatigue
  • Unfocused crowds = tacit interaction
  • Crowd = where the action is
  • World Trade Center…firefighters, follow routine of doing their job
  • Training not simply learning = establishing identity with the group who carry out their skills collectively.  Maintaining collective identity is an ongoing activity
  • Mandanity of excellence – routine, focused on their skills
  • What is a sacred object?
October 20, 2011 / B.J.D.Armas

Emotional Energy

by Randall Collins

  • Famous emotions = dramatic = fear, terror, anger, embarrassment, joy
  • Initiating ingredient = participants share common mood
  • Competing feelings drivn out by main group mling
  • Rhythmic ntrainment
  • Collectiv effervescenc
  • Moral solidartiy = acts of altruism
  • Failed interaction = breakdwon of solidarity that generates shame…if shamed individual becomes focus for interaction, restorative justice
  • Focus of power ritual = giving and taking orders, crucial item = showing respect for order-giving process = frontstage personality = cognitions = official
  • Ritualistic assent
  • Order-takers los EE when pwer ritual does not bring solidarity ritual
  • Symbols of upper-class = negative sacred objects
  • Micro-level – how successful is interaction ritual?  High level of collective effervescence?  Where is individual located as IR takes place?
  • Meso-level:  What time do peopl spend in each other’s physical presence – dimension of social density
  • Order-givers maintain and sometimes gain EE, order-takers lose it.
  • Being in a group raises emotional energy, experiencing marginality or exclusion lowers it
  • Emotional energy stored….cognitive….expctation of being able to dominat particular kinds of situations or to enact membership in particular groups
  • I will gt a good feeling of power or status if I interact with so-and-so
  • High solidarity = smooth-flowing rhythmic coordination
  • Winners = make selves focus, set expectation around self
  • Losers let winners become focus = sacred object, object which attention of group is focused
  • Dominant person cope
  • Anger = capacity to mobilize energy to overcome a barrier to one’s ongoing efforts Frijda 1986…amount of anger = proportional to amount of effort
  • Anger = explosive reaction against frustrations….truly powerful do not become angry —- get their way without it.  Express anger = expression of weakness.  If frustration is overwhelmingly strong, feeeling is fear, not anger
  • Fear = short-term negative emotion
  • Crying = fear, social call for help in distress
  • When person’s market position of interactions is on negative side, intense dramatic experiences are stored up and carried over as traumas — 131
  • Unconscious, trace level
  • Persons who dominat rituals gain EE— use to dominate future IRs
  • Key to stratification = inequalities in emotional energy
  • EE = continuum from enthusiasm, confidendence, initiative at the high end to passivity and depression at the low end
  • High EE = erect, firmly and smoothly
  • Low EE = shrinking, passive, hesitating, disjointed
October 20, 2011 / B.J.D.Armas

Rubbish-Power – Towards a Sociology fo the Rubbish Society by Martin O’Brien

  • We treat waste as if it was immaterial, apart from world we inhabit
  • What does a rich man keep that a poor man throws away?  Snot
  • muck, garage, trash, snot, filth, waste
  • Burn, bury, sell, swap, recycle, re-use reduce, minimize, detoxify, disown, mine, sort, co-dispose, dump
  • Rubbish = dynamic that drives social change
  • Local opposition to incinerators or nuclear storage
  • How come house is not submerged in junk?  Collecting, transporting and routing, storing, treating, burying, burning
  • Removal or dispersion of shared meanings = codified in hierarchical meanings and experiences
  • Wasting = translation, each person’s public/private location of waste objects translates industrial commodities thru a domestic waste stream into new kinds of industrial commodities
  • Where does your life go when you do die?  If poor — it is likely that your possessions…will be collected by a waste contractor and buried or burned in the local landfill or incinerator?  What tokens and records of yourself that you leave behind will be classified as waste in the common-sense meaning of the term:  as a superfluous, pointless nuisance.  If you are rich and connected some of your tokens and records will probably comprise historical archives:  monuments to the passage of time.  Their historical location and significance—their contents and their meaning– will pass into institutional memory.  Their historical location and significance — their contents and their meaning — will pass into institutional memory.  The social remanents that comprise the historical records of contemporary society did not ‘survive’ a psychological process of mass forgetting:  they simply avoided burial in the earth or burning in the flames.  Today, the power to burna nd bury other people’s histories is formulated in environmental departments and pursued in the name of environmental health and waste management.  Landfills are graveyards for the poor’s personal histories and incinerators their crematoria.
  • Toenails in your own home are not clinical wastes.  Toenails in an institution are clinical wastes.
  • Germany industrial economy — extraction of raw materials thru manufacture, distribution, consumption, disposal — waste or residue becomes first stage in process of manufacture
  • Elimination of household wastes = planning a minimum-waste cooking regime..
  • Men exploit depleting resources, women exchange fixed resources
  • Productive process = breach in the debris.
  • Existence of objects as goods = temporary social form as they pass from thestatus of waste to the status of waste.  The political economy of recylcing is thus the passage of an object not through a productive process again, but through a process of wastage again:  the return of the object to its concealed status.  The rubbish society is an apogenic society, a society whose means of permanently disposing of or removing waste creats its own opposite:  the appearance of a temporary value in industrial commodities.
  • Wasting is not the exit of value from systems of production and consumption.  It is a contested entry-exit-re-entry of economic and cultural values as systems of production, consumption and exchange.  Wasting, in fact, underpins such systems:  without the social process of wasting they could neither exist nor continue to operate:  rubbish is temporrally, socially, politically and economically dynamic.
  • Objects in a commodity phase are in phase of nuisance, utility, or beauty.
  • Plastic container = vehicle of commodification – supports the commodity of landfill – supported by the commodity of energy – exists in commodity context
  • Borough agrees to provide the commodity waste-fuel in exchange for protectionfrom the market
  • The commodity context of the plastic bottle as a waste-fuel is made condition on market protection and established temporarily
  • Plastic bottle that is a waste-commodity = beautiful attractive cultural form.  Beauty-bottle is located in a network of dependentobjects that substantiate its value but establish counter-discourse of rubbish
  • Junk = familial and spiritual object of desire
  • Junk = foundation of life, gives meaning an memory to personal, familial, and social change — cahgning constituents provide social and industrial markers of personal history
  • Junk = spiritual, may not be useful on the day-to-day but you want it around.  Junk is beautiful spiritual because its networks include “my song” and “getting older”, rather than the commodity values of energy and land.
  • Wasting = social process, embroiled in social meanings, practices, achieved thru networks and institutions, a process involving desire, spiritual and beauty as well as commodities, profits and resources