[RFC] Bug Bounty Program

Governance
#1

Given what has happened as of late, I believe it’s in the interest of the community to start a bug bounty program. However, as with most things, we want to hear your thoughts on how it should be structured.

I’ll get the ball rolling with some questions for y’all to answer:

  1. What projects have a bug bounty program we should learn from?
  2. How much $ from the Treasury should be devoted to this?
  3. How should we structure bugs classification (severity categorization etc.)?

Please add your own comments and further questions.

5 Likes
[PIP-18] Implement Bug Bounty Program
#2

A coin I like runs like this:

I think maybe 75% of the treasury should be on offer for critical bugs because any critical bugs would cost more in damages (i.e loss of funds, reputation etc), and it would leave us with 25% as an emergency fund ~ but I mean logically and rationally it could be even higher than 75%, but it leaves us in a precarious position if another bug or hiccup was around the corner.

I have zero expertise in this and should not be taken seriously if this is a bad idea!

1 Like
#3

I am very happy to see this RFC, hopefully this will encourage white hats to join our ranks. I want to outline a few items for those who might not be familiar.

This should probably fill any gaps moving forward so all can participate in the discussion.

There seems to be a lot to learn from the existing bug bounty created by yEarn. While our needs and resources are not quite the same the overall structure looks to be a good model especially regarding criticality of the bugs. I think there is an opportunity to use the same leveling, perhaps rewording some of the descriptions or tweaking the delineation slightly (i.e. cost prohibitive attacks maybe factoring in) although it may be better to stick with straight theory given we do not know an attackers resources.

As for an allocation on the bounty our treasury does not seem to be adequate to support what yEarn currently offers. I would propose creating a buffer for the bounty within the treasury to ensure funds are always on hand for such an issue. A total bug bounty of $200k (40% of yEarn) would be roughly 1/3 of our current treasury. This can be a starting point for discussion on what the allocation to this fund may be.

Given the payouts defined in the yEarn bounty document this should be sufficient to cover most issues that arise - unless the team anticipates otherwise.

Criticality Amount
Severe 4
High 10
Medium 40
Low 200

Above is the total bounties supported by a $200k fund for bounties which I think is very reasonable and may even be too high. As we have seen though security is a serious priority with recent incidents and we want to properly incentivize actors to report bugs rather than exploit them.

8 Likes
#4

I strongly agree with Jintao’s comment. A well-funded bug bounty program will not only make our protocol safer, but it will also (hopefully) attract talent to our little project. My technical expertise in these matters is very limited, so I leave working out the detail to those that can. Yearn seems a good example to follow.

I can only strongly echo the point Jintao made:

“As we have seen though security is a serious priority with recent incidents and we want to properly incentivize actors to report bugs rather than exploit them”.

It will also help with the public perception of Pickle.Finance. In times of regular bugs and rugs, we need to make sure people associate Pickle with strong security practices and due diligence.

1 Like
#5

We should definitely move to make the bug bounty program a reality. I support @jintao comment /agree

2 Likes
#6

Agreed on spending enough dough on downward risks first. Would do so in line with a wider risk assessment like e.g. devs leaving, community breakup, competition innovating faster etc. No expert here, just want to participate in building the DAO.
What are the chances for DeFi insurance on smart contracts (too expensive? no proper insurance on this yet?) or claims towards auditors (pipe dream? no idea).

1 Like
#7

Upfront disclaimer that I do not have any expertise in this matter, however some conclusions I have drawn from the recent events:

  1. It seems like the coverage of Smart Contract audits is not really economic exploitation vectors. This is as much i could gather from all the info I read on Harvest’s latest incident. Multiple audits from reputed auditors does not seem to really add value if its not able to scan for these vulnerabilities.
  2. Is it an idea to have like a hackathon event to actually stress test our strategies by offering an initial bounty for anyone that finds critical exploits like the one harvest had?
  3. In the industry i work, there is always an independent reviewer who has had nothing to do with the project, the review is generally done on critical design elements and the reviewer is someone that has earned that position by being extremely experienced in similar design. Would it be possible for Pickle and another well known project that are in a similar boat to offer each other dev resource time to actually look for vulnerabilities?

Thats just a mind dump. If it doesnt make much sense please ignore.

1 Like
#8

Welcome to the forum! Glad you decided to participate :slight_smile:

#9

This is great. I think it still a little on the pricey side. I would recommend giving a range of possible payouts per bug.

For learnings, see GitHub: https://bounty.github.com/

Mayhaps we can add cosmetic as a category too (under Low) to get everyone involved. Like reward the cosmetic bugs with 1 PICKLE. :smiley:

4 Likes
#10

Besides normal bug bounties, I’ll suggest we explicitly allow a hacker to walk away with 10% of the money they are able to steal. Getting hacked sucks, but if the hacker can return 90% and gets to keep the 10% without threat of repercussion, this seems like a win-win for all.

1 Like
#11

That’s an interesting approach … Isn’t the idea to indentify the bugs before theft? If the hacker had 100% then not sure they are motivated to return 90% because they are given permission …

1 Like
#12

Bug bounties tend to be insufficient reward for the work & skill required to find high caliber bugs, and even when they are high, they are awarded at the discretion of the company, which carries a risk of not getting paid fairly (from the PoV of the hacker). So the “rational decision” for a hacker is sometimes to exploit the vulnerability and run with the money.

By telling a hacker “you can hack us legally and you get to keep 10%” what I hope to accomplish is for a hacker to prefer the safer route where we’d only lose 10% of the funds, and the attacker gets to keep these funds without having to protect his identity, hide the money, launder it, etc, which also has huge risks and costs.

1 Like
#13

I agree! This was an arbitrary number that I just threw up and would have to hear what you think is more reasonable. Maybe instead we target 10% of the treasury so maybe $75k?

I realized another thing to consider is what is the bounty paid in? You mentioned PICKLE, however I think 3crv or something similar is likely a better option as it is linked to the dollar.

Rewards

  • Severe: 20,000-50,000 3crv
  • High: 5,000-20,000 3crv
  • Medium: 1,000-5,000 3crv
  • Low: 100-1,000 3crv
  • Cosmetic: 10-20 3crv

Another item to discuss is who is responsible for deciding criticality and qualification for bounty?

2 Likes
#14

@jintao I think we can pay the bounties in ETH or stablecoins (or, as you say, other dollar-linked assets since the dollar is the unit of account). I was mentioning 1 PICKLE only for the cosmetic category I proposed because it is more marketable to the community as a whole but it is not necessarily as good or convenient to come up with PICKLE to pay for the higher categories.

Basically, the criticality is a matter of describing the acceptance criteria for each tier as extensively as possible, and then paying well and fairly becomes good policy because all the community is watching.

I’d modify the guidelines to:

Rewards

  • Severe: $20,000-50,000+ `
  • High: $10,000-20,000
  • Medium: $5,000–10,000
  • Low: $100-5,000
  • Cosmetic: 1 PICKLE

But it all depends on what we mean by each tier.

3 Likes
#15

Should the severity be defined by the potential for loss? Severe = they could drain all or the majority of the TVL, etc?

I do love the idea of the Cosmetic rewards - a pickle here or there would go a long way to engaging the community.

2 Likes
#16

There are definitions that correspond to the levels in the yEarn document in my top level comment. Your assumption is correct though.

1 Like
#17

A lot of great info here guys, I’m going to try to prioritize this since we become a bigger and bigger target as our TVL grows.

3 Likes
#18

Let’s move forward with this model. But maybe without the cosmetic category for now), I’m hesitant on inviting 9999 cosmetic improvements like typos and small UX changes.

Since payouts will be done by the Treasury multi-sig holders, I suppose it will be whatever is convincing enough to 3/6 of the multi-sig holders.

2 Likes
#19

I agree with the amounts suggested by @leekuanjew and propose we move forward with that. I also agree that rewards should be denominated in stablecoin so as to attract the broadest range of people to our bounty program (some of which may not be interested in PICKLEs). However, while fun, I think we should eliminate the ‘Cosmetic’ category. There are invariably going to be a lot of those and it would take a lot of overhead to manage rewards for that category.

I see that the suggested amounts closely reflect Yearn’s and I agree with @jintao that there’s a lot we can learn from their bug bounty program. We will likely tweak some things as we flesh out our program, but I find their methodology for categorizing the criticality of identified bugs to be particularly helpful.

Here’s what I think we should move forward with:

Criticality Definition Amount
Severe Highly likely to have a material impact on availability, integrity, and/or loss of funds $20,000-50,000
High Likely to have impact on availability, integrity, and/or loss of funds $10,000-20,000
Medium Possible to have an impact on availability, integrity, and/or loss of funds $5,000–10,000
Low Unlikely to have a meaningful impact on availability, integrity, and/or loss of funds $100-5,000
3 Likes
#20

I have started working on a draft here:

3 Likes