Designing a study on outreach method

Vegan message board for support on vegan related issues and questions.
Topics include philosophy, activism, effective altruism, plant-based nutrition, and diet advice/discussion whether high carb, low carb (eco atkins/vegan keto) or anything in between.
Meat eater vs. Vegan debate welcome, but please keep it within debate topics.
Jamie in Chile
Senior Member
Posts: 399
Joined: Tue Apr 11, 2017 7:40 pm
Diet: Vegetarian

Re: Designing a study on outreach method

Post by Jamie in Chile »

I thought about this when I wrote the original post. A control group doesn't seem to be strictly necessary as the intent of the study is to show which appproach is relatively better, so it doesn't need an absolute baseline. Therefore no control group doesn't render the study useless, and at most limits its value. However the control group would be much cheaper and easier to run, no workshops and talks, so probably only adding 10% to the cost, so probably worth it as it does seemto add some value.
Jamie in Chile
Senior Member
Posts: 399
Joined: Tue Apr 11, 2017 7:40 pm
Diet: Vegetarian

Re: Designing a study on outreach method

Post by Jamie in Chile »

Having 1 person do all the workshops and talks is a good idea provided that person is very good and you're careful about who you choose. Yes, there is an advantage to it of avoiding bias (although having many people involved and rotating them should also eliminate any overall bias from a statistical standpoint). The person would need to have a lot of time available and be willing to travel around because I'm talking about a large sample size study at many locations. Maybe $100k is a bit high, but it depends on the sample size and I assume the person would get paid. If one person were willing to do all the talks and workshops for free it could be dramatically lower. I'm not sure who this person is or how they are making a living because this isn't going to be something most people could do in their spare time while working.
User avatar
Jebus
Master of the Forum
Posts: 2379
Joined: Fri Oct 03, 2014 2:08 pm
Diet: Vegan

Re: Designing a study on outreach method

Post by Jebus »

brimstoneSalad wrote: Wed Mar 07, 2018 5:49 pmIt's possible that an aggressive lecture could put people off going vegan who were already inclined to it, and might have done if they didn't hear that lecture at all.
The purpose of the study is to uncover if a soft approach is more effective than a harsh approach, and we would get that information by comparing the before and after effect in each group. If the study is carried out properly these are the different outcomes:

1. Both the harsh and the soft intervention methods are effective although one is more effective than the other. (In which case we can assume that both are better than doing nothing).
2. Both the harsh and the soft intervention methods are ineffective. One may be more ineffective than the other but who cares. We would then know that it is better to not do anything.
3. Neither approach has an impact. Common sense would then tell us that doing nothing is the best approach as it saves time (and potentially money).

There is also the unlikely possibility that the middle ground approach is the most successful. In this case, I would suspect a study contamination and recommend a repeat study. If the middle ground approach were still the most successful (or least unsuccessful) the crucial point would be whether or not the subjects felt more or less likely to become vegan after the intervention.
brimstoneSalad wrote: Wed Mar 07, 2018 5:49 pmDoing nothing at all, people may find their way to veganism on their own through exposure to these ideas, or just thinking about it.
The after questionnaire must be delivered rather soon after the intervention to avoid outside contaminants. As the proposed study is operating within such a narrow time frame, someone's probability to "find their way to veganism on their own through exposure to these ideas, or just thinking about it" would have to be dealt with within that own time frame.

Asking a group of people who have not seen (or heard) any intervention a few questions and then asking the same questions a couple of days later would just be a waste of time and resources.

A longitudinal study with a control group would also fail in terms of validity as it would fail to isolate the effect of the intervention.
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
User avatar
Jebus
Master of the Forum
Posts: 2379
Joined: Fri Oct 03, 2014 2:08 pm
Diet: Vegan

Re: Designing a study on outreach method

Post by Jebus »

Jamie in Chile wrote: Wed Mar 07, 2018 5:51 pm I thought about this when I wrote the original post. A control group doesn't seem to be strictly necessary as the intent of the study is to show which appproach is relatively better, so it doesn't need an absolute baseline. Therefore no control group doesn't render the study useless, and at most limits its value.
Just saw this after I wrote my previous post. Yes, this is exactly my point (in a more abbreviated delivery fashion).
Jamie in Chile wrote: Wed Mar 07, 2018 5:51 pmHowever the control group would be much cheaper and easier to run, no workshops and talks, so probably only adding 10% to the cost, so probably worth it as it does seem to add some value.
As you mentioned in your first post, such a study would need a large number of subjects. Anyone who has done research knows how difficult this can be. The control group would have to be as large as the other groups.
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
User avatar
brimstoneSalad
neither stone nor salad
Posts: 10273
Joined: Wed May 28, 2014 9:20 am
Diet: Vegan

Re: Designing a study on outreach method

Post by brimstoneSalad »

Jebus wrote: Wed Mar 07, 2018 11:43 pm 1. Both the harsh and the soft intervention methods are effective although one is more effective than the other. (In which case we can assume that both are better than doing nothing).
It wouldn't be able to tell us this. It could tell us they're the same, or that one is larger than the other.
In order to find out if they're effective, we need a control.

Let's say 10% of the people indicate they reduced meat consumption. What if that's what happens with this generation (we're talking about a college age sample) during this time without intervention?
Jebus wrote: Wed Mar 07, 2018 11:43 pm3. Neither approach has an impact. Common sense would then tell us that doing nothing is the best approach as it saves time (and potentially money).
What if the control reveals that this is a time when college students increase meat consumption for some reason? Or get into paleo?
And the intervention prevented that?
Jebus wrote: Wed Mar 07, 2018 11:43 pmThe after questionnaire must be delivered rather soon after the intervention to avoid outside contaminants. As the proposed study is operating within such a narrow time frame, someone's probability to "find their way to veganism on their own through exposure to these ideas, or just thinking about it" would have to be dealt with within that own time frame.
A tight time frame unfortunately gives them very little time for it to sink in.
We also have to contend with outside influences keeping them on course, because we have to check back in.

Given the lower cost of the control and how much it tells us, I think it's worth it.
User avatar
Jebus
Master of the Forum
Posts: 2379
Joined: Fri Oct 03, 2014 2:08 pm
Diet: Vegan

Re: Designing a study on outreach method

Post by Jebus »

brimstoneSalad wrote: Wed Mar 07, 2018 11:52 pm
Jebus wrote: Wed Mar 07, 2018 11:43 pm 1. Both the harsh and the soft intervention methods are effective although one is more effective than the other. (In which case we can assume that both are better than doing nothing).
It wouldn't be able to tell us this. It could tell us they're the same, or that one is larger than the other.
Huh??? I thought I covered the option of them being the same. What do you mean by larger?
brimstoneSalad wrote: Wed Mar 07, 2018 11:52 pmLet's say 10% of the people indicate they reduced meat consumption. What if that's what happens with this generation (we're talking about a college age sample) during this time without intervention?
This is irrelevant to the hypothesis at hand. Imagine an activist has decided that he will do a college tour speaking about veganism and he just wants to know if a harsh approach is more effective than a soft approach. This is what interests me. Whether or not not doing an intervention at all is more effective could be the hypothesis for a entirely different study.
brimstoneSalad wrote: Wed Mar 07, 2018 11:52 pmWhat if the control reveals that this is a time when college students increase meat consumption for some reason? Or get into paleo?
And the intervention prevented that?
Again, irrelevant to the original question that was brought up during the vegan daughter thread.
brimstoneSalad wrote: Wed Mar 07, 2018 11:52 pmA tight time frame unfortunately gives them very little time for it to sink in.
brimstoneSalad wrote: Wed Mar 07, 2018 11:52 pmWe also have to contend with outside influences
Right, it's a trade off. I think that waiting for the message to sink in is a bad idea. People often make decisions immediately after an intervention and by waiting we would be exposing the study to a bunch of contaminants. A follow up study (as I mentioned earlier) could be interesting (to see if those who were positively impacted by the intervention stuck to their guns).
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
User avatar
brimstoneSalad
neither stone nor salad
Posts: 10273
Joined: Wed May 28, 2014 9:20 am
Diet: Vegan

Re: Designing a study on outreach method

Post by brimstoneSalad »

One effect larger than the other. But without comparing to nothing, we don't know it has no effect, or negative effect, or positive.

Nothing may be naturally negative or positive.

"Imagine an activist has decided that he will do a college tour speaking about veganism and he just wants to know if a harsh approach is more effective than a soft approach."

It would tell us that, if that's all we want to know.

But I'd rather know the absolute magnitude so I could compare to other things too.
esquizofrenico
Junior Member
Posts: 71
Joined: Thu Feb 15, 2018 4:54 am
Diet: Vegan

Re: Designing a study on outreach method

Post by esquizofrenico »

The biggest problem I see with this kind of experiment is that you are judging your results from what people are willing to say, this could give a lot of problems. For example, it could be that a harsh approach make a lot of people angry that will say they will consume the same amount of meat or more, while to a few it causes a deep impact and makes them to really take the matter seriously; while a "nice" approach make people like the speaker and sympathetically say they will reduce their meat consumption, while actually not feeling any drive to do it in the real life. In that case this experiment would determine that the "nice" approach is more efficient, wrongly.

One of the greatest problems in determining the effectiveness of proselytism is slacktivism. There is people that are going to act fired up about a problem, while at the same time being completely unwilling to make any sacrifices to help solve it. In my opinion, any experiment needs to be specifically designed so that this group is distinguished from those that are really compromised with the topic. But of course, unless you spy your subjects that is extremely hard to know.
User avatar
Jebus
Master of the Forum
Posts: 2379
Joined: Fri Oct 03, 2014 2:08 pm
Diet: Vegan

Re: Designing a study on outreach method

Post by Jebus »

esquizofrenico wrote: Thu Mar 08, 2018 5:43 am The biggest problem I see with this kind of experiment is that you are judging your results from what people are willing to say, this could give a lot of problems. For example, it could be that a harsh approach make a lot of people angry that will say they will consume the same amount of meat or more, while to a few it causes a deep impact and makes them to really take the matter seriously; while a "nice" approach make people like the speaker and sympathetically say they will reduce their meat consumption, while actually not feeling any drive to do it in the real life. In that case this experiment would determine that the "nice" approach is more efficient, wrongly
I also thought about this. The only solution is to add a later follow up study which asks about behavioral changes.
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
carnap
Anti-Vegan Troll
Posts: 414
Joined: Wed Feb 01, 2017 12:54 pm

Re: Designing a study on outreach method

Post by carnap »

Jamie in Chile wrote: Wed Mar 07, 2018 5:51 pm I thought about this when I wrote the original post. A control group doesn't seem to be strictly necessary as the intent of the study is to show which appproach is relatively better, so it doesn't need an absolute baseline. Therefore no control group doesn't render the study useless, and at most limits its value.
Without a control group you couldn't know whether either strategy actually worked, only that one was worse than the other or they were comparable. That makes the study pretty useless because you wouldn't know whether either strategy was better than the baseline (e.g., doing nothing at all).
I'm here to exploit you schmucks into demonstrating the blatant anti-intellectualism in the vegan community and the reality of veganism. But I can do that with any user name.
Post Reply