Designing a study on outreach method
-
- Senior Member
- Posts: 399
- Joined: Tue Apr 11, 2017 7:40 pm
- Diet: Vegetarian
Re: Designing a study on outreach method
I thought about this when I wrote the original post. A control group doesn't seem to be strictly necessary as the intent of the study is to show which appproach is relatively better, so it doesn't need an absolute baseline. Therefore no control group doesn't render the study useless, and at most limits its value. However the control group would be much cheaper and easier to run, no workshops and talks, so probably only adding 10% to the cost, so probably worth it as it does seemto add some value.
-
- Senior Member
- Posts: 399
- Joined: Tue Apr 11, 2017 7:40 pm
- Diet: Vegetarian
Re: Designing a study on outreach method
Having 1 person do all the workshops and talks is a good idea provided that person is very good and you're careful about who you choose. Yes, there is an advantage to it of avoiding bias (although having many people involved and rotating them should also eliminate any overall bias from a statistical standpoint). The person would need to have a lot of time available and be willing to travel around because I'm talking about a large sample size study at many locations. Maybe $100k is a bit high, but it depends on the sample size and I assume the person would get paid. If one person were willing to do all the talks and workshops for free it could be dramatically lower. I'm not sure who this person is or how they are making a living because this isn't going to be something most people could do in their spare time while working.
- Jebus
- Master of the Forum
- Posts: 2379
- Joined: Fri Oct 03, 2014 2:08 pm
- Diet: Vegan
Re: Designing a study on outreach method
The purpose of the study is to uncover if a soft approach is more effective than a harsh approach, and we would get that information by comparing the before and after effect in each group. If the study is carried out properly these are the different outcomes:brimstoneSalad wrote: ↑Wed Mar 07, 2018 5:49 pmIt's possible that an aggressive lecture could put people off going vegan who were already inclined to it, and might have done if they didn't hear that lecture at all.
1. Both the harsh and the soft intervention methods are effective although one is more effective than the other. (In which case we can assume that both are better than doing nothing).
2. Both the harsh and the soft intervention methods are ineffective. One may be more ineffective than the other but who cares. We would then know that it is better to not do anything.
3. Neither approach has an impact. Common sense would then tell us that doing nothing is the best approach as it saves time (and potentially money).
There is also the unlikely possibility that the middle ground approach is the most successful. In this case, I would suspect a study contamination and recommend a repeat study. If the middle ground approach were still the most successful (or least unsuccessful) the crucial point would be whether or not the subjects felt more or less likely to become vegan after the intervention.
The after questionnaire must be delivered rather soon after the intervention to avoid outside contaminants. As the proposed study is operating within such a narrow time frame, someone's probability to "find their way to veganism on their own through exposure to these ideas, or just thinking about it" would have to be dealt with within that own time frame.brimstoneSalad wrote: ↑Wed Mar 07, 2018 5:49 pmDoing nothing at all, people may find their way to veganism on their own through exposure to these ideas, or just thinking about it.
Asking a group of people who have not seen (or heard) any intervention a few questions and then asking the same questions a couple of days later would just be a waste of time and resources.
A longitudinal study with a control group would also fail in terms of validity as it would fail to isolate the effect of the intervention.
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
- Jebus
- Master of the Forum
- Posts: 2379
- Joined: Fri Oct 03, 2014 2:08 pm
- Diet: Vegan
Re: Designing a study on outreach method
Just saw this after I wrote my previous post. Yes, this is exactly my point (in a more abbreviated delivery fashion).Jamie in Chile wrote: ↑Wed Mar 07, 2018 5:51 pm I thought about this when I wrote the original post. A control group doesn't seem to be strictly necessary as the intent of the study is to show which appproach is relatively better, so it doesn't need an absolute baseline. Therefore no control group doesn't render the study useless, and at most limits its value.
As you mentioned in your first post, such a study would need a large number of subjects. Anyone who has done research knows how difficult this can be. The control group would have to be as large as the other groups.Jamie in Chile wrote: ↑Wed Mar 07, 2018 5:51 pmHowever the control group would be much cheaper and easier to run, no workshops and talks, so probably only adding 10% to the cost, so probably worth it as it does seem to add some value.
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
- brimstoneSalad
- neither stone nor salad
- Posts: 10280
- Joined: Wed May 28, 2014 9:20 am
- Diet: Vegan
Re: Designing a study on outreach method
It wouldn't be able to tell us this. It could tell us they're the same, or that one is larger than the other.
In order to find out if they're effective, we need a control.
Let's say 10% of the people indicate they reduced meat consumption. What if that's what happens with this generation (we're talking about a college age sample) during this time without intervention?
What if the control reveals that this is a time when college students increase meat consumption for some reason? Or get into paleo?
And the intervention prevented that?
A tight time frame unfortunately gives them very little time for it to sink in.Jebus wrote: ↑Wed Mar 07, 2018 11:43 pmThe after questionnaire must be delivered rather soon after the intervention to avoid outside contaminants. As the proposed study is operating within such a narrow time frame, someone's probability to "find their way to veganism on their own through exposure to these ideas, or just thinking about it" would have to be dealt with within that own time frame.
We also have to contend with outside influences keeping them on course, because we have to check back in.
Given the lower cost of the control and how much it tells us, I think it's worth it.
- Jebus
- Master of the Forum
- Posts: 2379
- Joined: Fri Oct 03, 2014 2:08 pm
- Diet: Vegan
Re: Designing a study on outreach method
Huh??? I thought I covered the option of them being the same. What do you mean by larger?brimstoneSalad wrote: ↑Wed Mar 07, 2018 11:52 pmIt wouldn't be able to tell us this. It could tell us they're the same, or that one is larger than the other.
This is irrelevant to the hypothesis at hand. Imagine an activist has decided that he will do a college tour speaking about veganism and he just wants to know if a harsh approach is more effective than a soft approach. This is what interests me. Whether or not not doing an intervention at all is more effective could be the hypothesis for a entirely different study.brimstoneSalad wrote: ↑Wed Mar 07, 2018 11:52 pmLet's say 10% of the people indicate they reduced meat consumption. What if that's what happens with this generation (we're talking about a college age sample) during this time without intervention?
Again, irrelevant to the original question that was brought up during the vegan daughter thread.brimstoneSalad wrote: ↑Wed Mar 07, 2018 11:52 pmWhat if the control reveals that this is a time when college students increase meat consumption for some reason? Or get into paleo?
And the intervention prevented that?
brimstoneSalad wrote: ↑Wed Mar 07, 2018 11:52 pmA tight time frame unfortunately gives them very little time for it to sink in.
Right, it's a trade off. I think that waiting for the message to sink in is a bad idea. People often make decisions immediately after an intervention and by waiting we would be exposing the study to a bunch of contaminants. A follow up study (as I mentioned earlier) could be interesting (to see if those who were positively impacted by the intervention stuck to their guns).
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
- brimstoneSalad
- neither stone nor salad
- Posts: 10280
- Joined: Wed May 28, 2014 9:20 am
- Diet: Vegan
Re: Designing a study on outreach method
One effect larger than the other. But without comparing to nothing, we don't know it has no effect, or negative effect, or positive.
Nothing may be naturally negative or positive.
"Imagine an activist has decided that he will do a college tour speaking about veganism and he just wants to know if a harsh approach is more effective than a soft approach."
It would tell us that, if that's all we want to know.
But I'd rather know the absolute magnitude so I could compare to other things too.
Nothing may be naturally negative or positive.
"Imagine an activist has decided that he will do a college tour speaking about veganism and he just wants to know if a harsh approach is more effective than a soft approach."
It would tell us that, if that's all we want to know.
But I'd rather know the absolute magnitude so I could compare to other things too.
-
- Junior Member
- Posts: 71
- Joined: Thu Feb 15, 2018 4:54 am
- Diet: Vegan
Re: Designing a study on outreach method
The biggest problem I see with this kind of experiment is that you are judging your results from what people are willing to say, this could give a lot of problems. For example, it could be that a harsh approach make a lot of people angry that will say they will consume the same amount of meat or more, while to a few it causes a deep impact and makes them to really take the matter seriously; while a "nice" approach make people like the speaker and sympathetically say they will reduce their meat consumption, while actually not feeling any drive to do it in the real life. In that case this experiment would determine that the "nice" approach is more efficient, wrongly.
One of the greatest problems in determining the effectiveness of proselytism is slacktivism. There is people that are going to act fired up about a problem, while at the same time being completely unwilling to make any sacrifices to help solve it. In my opinion, any experiment needs to be specifically designed so that this group is distinguished from those that are really compromised with the topic. But of course, unless you spy your subjects that is extremely hard to know.
One of the greatest problems in determining the effectiveness of proselytism is slacktivism. There is people that are going to act fired up about a problem, while at the same time being completely unwilling to make any sacrifices to help solve it. In my opinion, any experiment needs to be specifically designed so that this group is distinguished from those that are really compromised with the topic. But of course, unless you spy your subjects that is extremely hard to know.
- Jebus
- Master of the Forum
- Posts: 2379
- Joined: Fri Oct 03, 2014 2:08 pm
- Diet: Vegan
Re: Designing a study on outreach method
I also thought about this. The only solution is to add a later follow up study which asks about behavioral changes.esquizofrenico wrote: ↑Thu Mar 08, 2018 5:43 am The biggest problem I see with this kind of experiment is that you are judging your results from what people are willing to say, this could give a lot of problems. For example, it could be that a harsh approach make a lot of people angry that will say they will consume the same amount of meat or more, while to a few it causes a deep impact and makes them to really take the matter seriously; while a "nice" approach make people like the speaker and sympathetically say they will reduce their meat consumption, while actually not feeling any drive to do it in the real life. In that case this experiment would determine that the "nice" approach is more efficient, wrongly
How to become vegan in 4.5 hours:
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
1.Watch Forks over Knives (Health)
2.Watch Cowspiracy (Environment)
3. Watch Earthlings (Ethics)
Congratulations, unless you are a complete idiot you are now a vegan.
-
- Anti-Vegan Troll
- Posts: 414
- Joined: Wed Feb 01, 2017 12:54 pm
Re: Designing a study on outreach method
Without a control group you couldn't know whether either strategy actually worked, only that one was worse than the other or they were comparable. That makes the study pretty useless because you wouldn't know whether either strategy was better than the baseline (e.g., doing nothing at all).Jamie in Chile wrote: ↑Wed Mar 07, 2018 5:51 pm I thought about this when I wrote the original post. A control group doesn't seem to be strictly necessary as the intent of the study is to show which appproach is relatively better, so it doesn't need an absolute baseline. Therefore no control group doesn't render the study useless, and at most limits its value.
I'm here to exploit you schmucks into demonstrating the blatant anti-intellectualism in the vegan community and the reality of veganism. But I can do that with any user name.