An Algorithm Question

C

CrazYmonkeY159

Expert Member
Joined
Sep 13, 2007
Messages
2,142
Reaction score
0
Location
CPT/PE
Hi

My question is related to voting and how to produce a result from the following bottom problem

So I want users to vote on certain things, the outcome has to be 1 singular result. The catch is that voters have associated 'confidence' so confidence can be defined as a percentage, so a voter with a confidence of 80% is 80% sure that the vote that they are making is correct. All easy so far? good

so imagine you have X amount of voters (x1, x2, .. , xn) each with their associated confidences C (c1, c2, .. , cn), they are voting on a choice. so there can be a Y amount of choice (y1, y2, ... ,ym)

My question is, is there an algorithm or way of coming up with the correct output given X and C and Y

I was told taking the most confident voter is not always correct. so if Voter 1 was 70% sure that Choice y4 was correct, but Voter 2 and 3 are 69% sure that y3 is the right vote. then Choice y3 should be the correct answer

I dont know if im being vague or incomplete in my question. any help would be appreciated. Ive looked at how some voting systems work, and none explain how they judge votes.
 
Last edited:
So is this like who wants to be a millionaire's ask the audience? One of the Y answers is correct, but the voters are unsure which one. And you have to pick which Y the audience, on average, thinks is correct based on their input and confidence weightings. Right?

Is that choice Y3 should be the correct answer, or could be the correct answer?

If its should, then you weight the numbers according to how many people voted and what their percentages were.

Ie, percentage for Y3:
C2 = .69
C3 = .69
Total C for Y3 = .69 + .69 = 1.38

Percentage for Y4:
C1 = 0.70
Total C for Y4 = .70

So Y3 wins in the case. Just add up all the C scores per Y. That will take into account not only how many people voted, but what their confidence scores were.
 
Just a question, can you only give one answer with your confidence, or multiple?

From your question it seems that you may only choose one answer, and i'm assuming it should be the one you have with highest confidence?
 
Ancalagon said:
So is this like who wants to be a millionaire's ask the audience? One of the Y answers is correct, but the voters are unsure which one. And you have to pick which Y the audience, on average, thinks is correct based on their input and confidence weightings. Right?

Is that choice Y3 should be the correct answer, or could be the correct answer?

If its should, then you weight the numbers according to how many people voted and what their percentages were.

Ie, percentage for Y3:
C2 = .69
C3 = .69
Total C for Y3 = .69 + .69 = 1.38

Percentage for Y4:
C1 = 0.70
Total C for Y4 = .70

So Y3 wins in the case. Just add up all the C scores per Y. That will take into account not only how many people voted, but what their confidence scores were.
Click to expand...

Yup this is the easiest way just consider the confidence as points and tally them up, highest score wins.
 
Faultyboy said:
Yup this is the easiest way just consider the confidence as points and tally them up, highest score wins.
Click to expand...

I think there is a flaw with this though..

If I know that Y4 is correct with 100% confidence
and the other 2 think Y3 is correct with 51% each
then Y3 will win..
 
According to the OP, that doesnt matter, because the single most confident voter is not always correct. And I'm assuming, thats even if he is 100% confident.
 
nakedpeanut said:
I think there is a flaw with this though..

If I know that Y4 is correct with 100% confidence
and the other 2 think Y3 is correct with 51% each
then Y3 will win..
Click to expand...

yes because you still need to consider how many people are voting. If 5 people vote 100% 50 people voting 11% will be needed to out weigh them. It gives the votes with less confidence less weight.

I don't like the system, a vote is a vote. You make your decision based on the one you are most confident about. The confidence should just be used for statistics not weighing against the vote.
 
Last edited:
To me it would make more sense if:
X1 -70%-> Y3
X2 -69%-> Y4
X3 -69%-> Y4
X4 -50%-> Y3
X5 -60%-> Y4

Thus
Y3 = 0.69+0.69+0.6 = 1.98 / 3(voters) = 0.66
Y4 = 0.7+0.5 = 1.2 / 2(voters) = 0.6

Thus Y3 wins.


Edit:
This gives the average confidence for each answer.
So I suppose you actually lose information with this.
In the end I suppose it's what you trying to achieve.

Faultyboy said:
yes because you still need to consider how many people are voting. If 5 people vote 100% 50 people voting 11% will be needed to out weigh them. It gives the votes with less confidence less weight.

I don't like the system, a vote is a vote. You make your decision based on the one you are most confident about. The confidence should just be used for statistics not weighing against the vote.
Click to expand...
With this I suppose it is true that Y3 got more votes.. And with a bigger sample of votes, multiple low confidence votes would be beat by fewer high confidence votes.. So I agree
 
Last edited:
nakedpeanut said:
To me it would make more sense if:
X1 -70%-> Y3
X2 -69%-> Y4
X3 -69%-> Y4
X4 -50%-> Y3
X5 -60%-> Y4

Thus
Y3 = 0.69+0.69+0.6 = 1.98 / 3(voters) = 0.66
Y4 = 0.7+0.5 = 1.2 / 2(voters) = 0.6

Thus Y3 wins.


Edit:
This gives the average confidence for each answer.
So I suppose you actually lose information with this.
In the end I suppose it's what you trying to achieve.


With this I suppose it is true that Y3 got more votes.. And with a bigger sample of votes, multiple low confidence votes would be beat by fewer high confidence votes.. So I agree
Click to expand...

Yes. You'd incorporate weightings to ensure that 100 votes of 99% doesn't get beaten by 1 vote of 100% (since by your calculation (100*0.99)/100 = 0.99 and 1*1 / 1 = 1)
 
What happens when your choice is not available? Then no matter what you vote for you won't be putting as much confidence into it. Straight away putting that vote at a disadvantage.

Yeah I think the fundamental problem with a system like this is the fact that not every vote is equal.

You can't have one vote out weighing another, also using the confidence in this way is flawed because you can't guarantee it's the persons true confidence which allows for manipulation. Every vote should be equal.

I think a better angle for the project would be to aim at designing a system that forces transparency and doesn't allow for vote manipulation.
 
Last edited:
Remember this doesn't just apply to your "confidence" in an answer or choice.

From a CRM stand-point, if User A has a proven track record (however you want to define it, years of service/number of sales made etc), and User B (a total n00b with almost no sales and/or years of service) and both vote on a choice, then choice of User A would weigh against User B if there's a 50/50 split, negating that of a stale-mate when it comes to a choice.

Keeping that in mind. If you have 5 people and 5 choices and it happens that all 5 people choose a unique choice, those that carry more "weight" (or confidence, however that is measured), pushes the voting system into showing a winner.

Someone equated it as the "Who wants to be a millionare" thing. I highly doubt confidence or weight carries in that and that it's a pure "1 vote = 1 vote" system as I've seen some answers have similar % outcomes before which should be rare if there's "confidence" factored into it.
 
AcidRaZor said:
From a CRM stand-point, if User A has a proven track record (however you want to define it, years of service/number of sales made etc), and User B (a total n00b with almost no sales and/or years of service) and both vote on a choice, then choice of User A would weigh against User B if there's a 50/50 split, negating that of a stale-mate when it comes to a choice.
Click to expand...

I agree, It would work if everyone's 'confidence' was obtained from a reliable system, but if the voter gets to determine the confidence when he votes it will fail.
 
nakedpeanut said:
Just a question, can you only give one answer with your confidence, or multiple?

From your question it seems that you may only choose one answer, and i'm assuming it should be the one you have with highest confidence?
Click to expand...

Yeah assume no stale mates/multiple outcomes, there has to be one answer. Im trying to implement this kinda system, at this point in time I think im doing a gross approximation/simple answer to this (with 'OK' results) I just need to find a more robust way of doing it. Thanks guys you gave me a lot of ideas though.
 
Who determines the confidence level? The Users themselves or the system (based on various criteria)?
Actually, to satisfy my curiosity, what is your real-world application for this algorithm? Perhaps there's a better/different/less complex way of getting a more satisfactory and robust result.
 
@AntiThesis

I'll be honest with you, im trying to improve on on results that I'm getting with my neural network, I read a paper where a person used 5-100 different neural networks which votes on a single query and tries to classify the query as being in class A,B,C etc

Confidence is basically how confident the classifier is with its own choice (which is kinda simple to calculate), you can improve classification by using this method, I think the technical term is called an Ensemble of Classifiers.
 
Could you tell us more about your Neural Network, Training methods, How many neurons, Did you code it your self or are you running MATLAB?

Have you run any genetic algorithms through the network for training and testing?
 
Last edited:
I didn't read all the replies but here's my input:

Why not just add the confidence as a weight to the option? Then you can use the principle of a weighted average.

So if user 1 picks A, you have 1 vote of 0.8 confidence, so a score of 0.8 for A.

If 3 users pick B with 2 confidences of 0.5 and one of 0.4, you'll have a score of 0.46 for B.
 
@ Baron, I think NakedPeanut mentioned this earlier. It doesn't really work out.
 
Faultyboy said:
@ Baron, I think NakedPeanut mentioned this earlier. It doesn't really work out.
Click to expand...

With a large amount of votes (> 100 for each option) it should work out nicely.

If you have 5-10 votes for each option, then the outliers will fsck it up.
 
Yeah larger number might smooth it out nicely, He will have to run the test.. :)

Now that I know this is for a neural network the 'confidence' makes a lot more sense.
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X