After winning DEF CON's annual Capture The Flag (CTF) competition five of the last seven years, the Plaid Parliament of Pwning (PPP) returns as the reigning champions during very different conditions because of COVID 19. How is the team preparing?
This second podcast from The Hacker Mind tells the story of one of PPP's members, Zaratec: How she first joined PPP, what skills she’s learned from CTFs in general and now uses in her real-world job, and the team is making changes for this year's online CTF final.
Listen to The Hacker Mind EP 02: Inside DEF CON’s Champion CTF Team PPP
Never miss another episode. Subscribe to The Hacker Mind podcast on:
Transcript: The Hacker Mind EP 02: Inside DEF CON’s Champion CTF Team PPP
Host: Robert Vamosi
Guest: Zaratec, PPP team member
Vamosi: So it’s, what, July and for me that would mean my 20th year going to Las Vegas...for Hacker Summer Camp, for Bsides Las Vegas, for Black Hat, and, of course, DEF CON. [Scratch/Rewind sound] Except I’m not going this year. No one is. Because of COVID-19, BSides Las Vegas is canceled, Black Hat is online, and DEF CON, also online, is in Safe Mode, a clever play on an operating system in diagnostic mode.
That means the annual DEF CON Capture the Flag competition, the World Cup of Hacking, is also online. And last year’s defending champion, PPP, aka the Plaid Parliament of Pwning, will have to compete online, of course. Why wouldn’t they? PPP has won five of the last seven years. Given their amazing record, you’re probably wondering just who are these elite super hackers? What mad skills does one have to possess to be on their team? And how has COVID-19 changed anything?
Welcome to the Hacker Mind, an original podcast from ForAllSecure. It's about challenging our expectations about people who hack for a living. I’m Robert Vamosi and this week I’m celebrating a virtual Hacker Summer Camp 2020 with an inside look at the number one capture the flag team in the world today, PPP. PPP is a competitive hacking team out of Carnegie Mellon University in Pittsburgh, and was formed around 2009, under the direction of Dr. David Brumley. It was created because one of Brumley’s students wanted to compete in Capture the Flag competitions, but rather than look outside for talent, they formed a team internally at CMU. So, right there, it seems that to join PPP you need to be a student at CMU. That also means you have to start early, you have to get good grades in high school and I suppose you should have already been exposed to computer hacking at an early age.
Zaratec: So back in middle school, I was fortunate enough to be in a I guess it was a magnet program, like a STEM program. And so I had a lot of experience kind of doing CS tech stuff for a while.
Vamosi: This is Zaratec. For her, computers have been a way of life since she was a pre-teen. And it really helped that the school she attended had a pathway for her to develop her computer science skills, and a teacher who recognized that potential early on.
Zaratec: So by the time I got to high school, I kind of felt very on top of like a lot of tech stuff and I kind of wanted to expand out and explore more. My computer science teacher at the time was very encouraging I guess to kind of try different things. And so he founds this competition called CSAW HSF which is high school forensics. I think it's under a different name now. I don't recall what it is, but essentially he was like, "You should go try that." And I like looked at it I was like, oh it's hacking stuff you know that sounds pretty cool.
Vamosi: CSAW is a well-established CTF competition in New York. It’s sponsored by the NYU Center for Cybersecurity. It bills itself as the most comprehensive student-run cyber security event in the world, featuring nine individual hacking competitions, including CTFs. It is held conveniently over the course of one weekend.
Zaratec: But it was I don't really know how to hack. I know how to do basic programming, you know, since I've had some experience in it, but at the time I was just like, you know as cool as it sounds I have no idea what I'm getting myself into. But I did it anyways. So, yeah, I tried doing it. I think he assigned some seniors to work on it but they quickly lost interest, so I ended up just doing it by myself. The way that this competition was structured was like a crime scene, so they had the storyline -- it was a play on Jersey Shore -- where Snooki or one of the other characters -- I have no idea. I don't watch these TV shows -- got killed and someone else was being framed and you have to figure out what was going on. There was some USB drive that was encrypted that they found so that's the only clue that you get. So you would have to go and figure out first of all that it's encrypted and then decrypted. Then there's a file in there that is also encrypted. You have to decrypt that as well. There was a steganography in the image. I just found it very fascinating that it, you kind of had to figure things out for yourself and it was always a puzzle that you had to piece together. It was just so much fun.
Vamosi: That does sound like fun. Like a reading a good mystery, except you are the main character trying to figure out all the clues.
Zaratec: And my CS teacher, he would recount to me later on he said, "every single day you were coming in with a sparkle in your eyes, telling me the latest things that you were finding." I was hooked from there.
Vamosi: Zaratec wasn’t alone in solving these puzzles. Remember she attended an advanced high school, full of smart kids.
Zaratec: So during high school I found some other folks that were also interested in this stuff and played a lot of different, mostly forensics type competitions. But sometimes we would go more into CTF, which were varied in terms of the different types of challenges that you would face. So, it wasn't just friends it was other stuff. And we did pretty well in them. By that time, I started thinking about college at one of these CSAW competitions in New York City. I saw that there was this team called PPP that was absolutely crushing everyone. So, as a high schooler I'm trying to figure out where I want to go. I did some digging and between that experience, seeing PPP just absolutely doing amazing and then also dabbling a little bit in PicoCTF.
Vamosi: PicoCTF is a smaller online CTF run by CMU. It’s designed for high students like Zaratec.
Zaratec: I was like, "okay CMU is a good place to go because this team does so well at this thing that I'm really interested in." And yeah that's essentially how I decided I want to go to CMU. I applied and I got in and I was like, "hello!"
Vamosi: So as a freshman college student, Zaratec just walked up and joined PPP, a team that has just won the CTF at DEF CON the previous year … okay, maybe it was a bit harder than that.
Zaratec: Yeah, it's been so long but I'm pretty sure as soon as I showed up to college I tried to figure out how to go to these PPP meetings and as soon as I could, I'd started going to them.
Vamosi: Wait. So PPP was, like, underground?
Zaratec: No, they were not underground. Usually the way that they would advertise them for the freshmen that are coming in, is they would put out flyers about a little ice cream social. So they draw people in with the ice cream social and then they socialize and talk about the CTF. From there most of the meetings are just normal CTF stuff and solving problems. But I either found one of those flyers, or I did some digging on -- I don't know -- one of the many Facebook groups that CMU had at that time and found out information. But now they hang out in silence at the cyber security lab. We're fortunate enough to usually be given a meeting room, that we can use for our own meetings.
Vamosi: Cylab is one of the largest university-based cybersecurity research and education institutes in the world, and it’s based out of CMU. It includes more than 50 faculty and 100 graduate students from different departments and schools within the university. So getting meeting space from them is a pretty big deal. And it was probably a good space for the team to prepare for DEF CON, although that wasn’t top of mind with Zaratec.
Zaratec: I'm pretty sure they had been going to DEF CON by then. Okay. I feel like my memory is a little fuzzy but I feel like they probably would have had one or two wins under their belt by that time. DEF CON for me wasn't something that I really learned the importance of until maybe sophomore year, or after I spent more time with PPP. For me it was just like: these guys play a lot of CTFs -- which CTF didn't matter to me -- just that they did a lot of CTFs. They seem very good.
Vamosi: It’s worth noting that if you want to join PPP, you should probably just play it cool. Really. You should get to know the team members one on one, but you should also be really, really good at hacking. Also, not everyone on the team competes in the CTF at DEF CON.
Zaratec: I think by the time I got to go to DEF CON. I truly understood the importance. But when I first joined the team, it wasn't because these are the DEF CON guys. It was more like, oh these guys are just very good at CTFs in general.
Vamosi: There are other CTFs, in fact there’s probably one every week. And there are other teams, hundreds of them. So how do the best teams get to compete at DEF CON each year? It starts with a round of qualifiers or quals in May or June. This is how the organizing committee determines who will be among the final contestants in July and August. However, the timing doesn’t always work for college students.
Zaratec: Qualifiers are a little difficult sometimes because of the timing. Historically the timing has usually overlapped with either finals or graduation. Move outs, final exams. Some combination. So historically for the students on the team, it's been difficult to participate. Now that being said that doesn't, of course, stop, a lot of the students, but a lot of the qualification rounds sometimes are played more heavily by the graduate students who have graduated, rather than the current students, which I think is flip flopped for most other CTFs. If the timing is right then, most people play. I think this year it actually worked out well, particularly because of coronavirus everyone was also home so it wasn't like you had obligations to be out and about.
Vamosi: The live event at DEF CON is an intense 72-hour hacking spree, with maybe a dozen teams made up of students, industry workers and government contractors all attempting to defend their own while breaking into each other’s systems, each stealing virtual “flags”, and accumulating points on a big board for all to see. On the second day, those points are hidden, and on the third day, even the team ranking is hidden so the teams have no idea how they are doing relative to the others. In 2016, PPP competed against 16 teams from 7 countries, some teams having about one hundred members. But Zaratec says having more team members doesn’t mean you’re more efficient or better.
Zaratec: When we're working on problems for instance, if you have 100 people working on a single problem, there's a bottleneck. How many people can efficiently work on that problem, right? But there's also the issue of, you don't want people to be duplicating work, especially in something like DEF CON where you are dealing with many many different problems, some of which are live, essentially, because they're attack, defense. So you need to watch what's going on, constantly. You really don't want to be duplicating work so there is a threshold of people that when you get past, you start having issues. People are duplicating work or maybe you have some people that don't really know what's going on, or where they would fit in best. And then that becomes an issue because they don't feel like they're contributing much, right, and we want people to feel like they're contributing. For us, I think we're probably one of the smaller teams that usually attend DEF CON. If not, one of the smallest consistently. We usually bring between 20 to 25. That's where we've been floating around the past couple of years. The reason for that is because we found that this is the number of participants where everyone that is attending is able to find something to do, and is very efficient with what they're doing such that we're making good use of our time. In the past, you had the team on the ground in Las Vegas, and you also had the ability to have a team remote, somewhere else. Of course this year everyone will be remote. That being said, I think we also offer the ability for people to play remotely, although that becomes a little difficult because of how the attack, defense is set up. Remote contributions end up being like carving out a chunk of a challenge and shipping it off to them and having them work on it. I don't think we've had very much remote participation. Mostly because it's also very hard to work on problems when you're not with the team. Part of why we're so effective at what we do is because of that team camaraderie. The way that we decide these teams is mostly by how much effort they been putting into CTF. How much have they been participating? If they've been making CTF problems for plaid CTF, have they historically also gone to DEF CON? If they have a lot of DEF CON experience, they're going to know exactly right off the bat, how to handle the DEF CON experience and how to make good use of their time. Newcomers typically will show up and maybe spend a day or so adjusting to figure out where they fit in. Because there's so many different things that you could do, it is just a very indecisive factor like, "oh I could fit in here or here so as it's the best use of my time and for the team also." But yeah, and so that's just an overview of how we have that team of finals.
Vamosi: Okay, DEF CON is the most crowded conference I have ever attended, and there are usually goons -- volunteers -- in the hallways just directing traffic and queuing up lines of people to enter certain rooms. And there are a lot of rooms. There’s the talks, of course, the different villages with their own talks, the vendor room, but also there’s this big room that usually has several competitions going on all at once. The main competition, though, is the CTF and there you will see a board with the teams’ names and any points they have -- at least on day one you’ll see that. You’ll also see various tables with the team logo or colors, with various people staring at their sticker covered laptops, trying to ignore everyone staring at them. Zaratec says that’s not the whole story.
Zaratec: For most of the teams, the tables are the tip of the iceberg. Some of the teams actually prefer not to be in the room. It's an inside joke that everyone absolutely hates that room because it's noisy. There's people that are coming and going, that are watching you. It's kind of like you know, an aquarium when you're the fish. And there's loud music and there was one year where they had just straight memes playing the whole time. So it's very distracting for people to be playing there. That being said, you need to have at least one person there to relay information and challenges to the rest of your team. How most teams have it set up is they buy a suite in one of the hotels, and they have it set up so that most of their team is there. That's at least what our team does. We have people on the floor that would be useful in terms of like they're the captain of the team, so they can go and talk to the organizers if need be. People that are senior and are able to deal with the noise and chaos of the downstairs area. And then also people that are good at creating tools or pipelining information that is given to us so for instance. Usually, sometimes, we are given packet captures so network data. Some of our teammates are very very very good at creating tools that are able to efficiently take up network data, and to kind of look through it very quickly to find important information which might be flags or exploits being thrown us thrown at us or like other things. Those folks are also very good to have downstairs just because they are sitting in the area where that data is like being given, I think. So we primarily use slack to communicate, uh, kind of, we're also moving to discord Now that everything is kind of being remote, because we found that having, you know, only to like text only stuff is, it's like pretty good, you know if your team is like close to each other or at least you're gonna see each other like once in a while like we are at DEF CON discord we found like in terms of like either seeing like having audio or video is really good for like this sort of situation, or just, you know, for the alumni who don't all live together, to kind of give that team camaraderie Greg because you can hear your your your teammates, you can speak with them it's so much easier to communicate some stuff over audio, and it just like feels a lot more natural.
Vamosi: So really it’s a question of being organized, using the resources you have in the moment. So you not only have to know how to hack, but how to prioritize and deputize members of the team.
Zaratec: We might actually do that this year some stuff con will be remote. But yeah, so usually as soon as the challenge drops, we will send that information to our teammates in the suite. And then people that are interested in picking up that challenge will say, I'm working on that challenge just so everyone else knows and like you know you don't have like 10 people dog piling on the same challenge. We want to kind of like spread people out to make sure that, you know, we're looking at all the challenges and working on them. Uh, I think that's mostly it for like the team, like how we work with that sort of setup and then of course like at night, they close off the area, so we all go back to the suite and we all work together, we'll usually get like food and we'll you know all eat it together and we'll just kind of like group up based on like you know oh I'm working on that challenge or they're working on that challenge and so people kind of group up and they work together. And then when the next day comes, we have like a battle plan of mines we have our exploits that we want to throw the people that go downstairs go downstairs and then they are the ones that also will do like the some of the throwing and whatnot and the people upstairs will just keep working on the challenges.
Vamosi: When the teams first arrive in Las Vegas, they have some information for the CTF, but since it’s also attack and defend you can never be fully prepared. That means while they are solving a puzzle, another team or teams could be attacking them at the same time, so they have to defend themselves.
Zaratec: We do have some information so DEF CON CTF has always been an attack, defense CTF. The format of that sometimes slightly changes. They might throw in like some twist to it. But the core structure of it has always been attack, defense, without information in mind we know that there are certain things one is that we need the ability to be able to watch whatever challenges or boxes that were given to defend them properly, and the other is we need something to be able to throw attacks at other competitors. In the case that we are given network captures of everything that's going into our boxes or challenges. We also need the ability to kind of quickly scan through those network captures, and get information from them as I mentioned before. So, those are like the core things. Other things that might you might find useful for instance, are just like different types of exploits. Can we make tools that will make it a lot easier to to do those exploits,
Continuous Testing at the Speed of Development.
Find out how ForAllSecure can bring advanced fuzz testing into your development pipelines.
Vamosi: With attack and defend, you need tools. You need to prepare something in advance, right?
Zaratec: Primarily, we used to actually focus a lot of time on prepping tools. I think that is done less so now, now that we have a new set of organizers. The previous set of organizers would give a lot of packet captures and network data, and we made tools that were strongly targeted towards that the new set of organizers doesn't really don't really give a lot of that information away. So we find it less useful to work on those sort of tools now. I think actually like the first year, we made a lot of tools that were targeted towards that and then the organizers didn't give any network captures and we were very very very sad, because we spent a lot of time preparing for that. Um, I think because of that we're a little more hesitant on prepping tools that far in advance. Usually we would do this like at the beginning of the summer, but because things are kind of like in flux, it's just really hard to put in a lot of work into something that you don't know like if it'll pay off at all. I think one thing that we've had schooling is like a thrower box of some sort. I'm not really sure the details of this because some of the older folks on the team have set it up probably like one of the first times they went to DEF CON and have been using ever since. But it's just like, essentially, some setup that allows us to quickly like throw exploits at other teams as soon as we know what we want to throw. I think it's been like everything from some decked out MacBook if I ever powerbook if I'm not mistaken, which highlight these crazy lights on the back or something and probably look fantastic. I think we stopped to bring them that, I'm not sure why, but it was pretty cool. Nowadays I think we just bring a knock or something like a small like computer. Yeah, I'm kind of wish we went back to the lights honestly like that sounds pretty cool.
Vamosi: Being on a CTF team at DEF CON is all consuming. I'm able to go to the talks, I'm able to walk around, meet up with friends, do some fun stuff. But if you’re on a CTF team...
Zaratec: I have never done anything else at DEF CON and I've been a three times I think I'm sure many of the other. Many of my other teammates I'm pretty sure we're in the same boat. Um, I think, like the most I've experienced is sitting on the CTF floor and then hearing like the other competitions going on, or maybe walking by the tinfoil hat contest or the pac, pac fortress events. It's a little unfortunate because you know there's a lot of cool stuff there. But realistically I think like if we are doing the CTF we are doing the CTF and we are going to put like all our focus in on that. And if you need a break and you want to like go do something else for a little bit that's fine but I think like everyone on the team is like super dedicated to doing their best on the CTF. And so every, I think most people just spend their whole time doing that.
Vamosi: Zaratec has since graduated CMU. She’s out in the real world today. But she’s still competing with PPP.
Zaratec: I think like jokingly we all say like what a PPP for life. Um, I think, you know, as you get older and you move away from college you have, you know, in college, you have your classes your homework and assignments and whatever you can kind of like do those quickly or you know shove them aside and procrastinate and then do CTFs all weekend. Now as you graduate from college and you now are like in a workforce and you know you have like an actual life, I guess, once again, it becomes a little more difficult to do CTFs, and especially because you know you're not with your friends and your teammates and it's just, it feels a little lonely sometimes and it becomes a lot more hard to communicate more complex ideas, especially, especially when you're working on challenges. That being said, I think everyone on the team is like super excited to be on the team. I think the knowledge that we learned is like incredible, the friendships that we have on the team are also like great. And I think everyone is just like super excited to play DEF CON like even if they maybe don't show up to, CTFs, like, super often during the year, DEF CON is like the one thing that everyone is usually like okay we're gonna go and do like super well and meet up and stuff. Also because we get to see each other like at least once a year, other time being played CTF. The CTF that we run. Um, but yeah, and I think things like discord and, you know, anything else that you know bolts audio and video has been like helping a lot, it makes people a lot more interested in playing because they can see their teammates and they can, you know, pack pretty efficiently.
Vamosi: Given her experience, Zaratec has some advice for those just starting out.
Zaratec: I'd say it's like their first time going into DEF CON and they're going into an established team. I think the hardest thing for the beginners is figuring out where do they fit in. Because you show up and you have a team of like, I don't know anywhere from 20, like our team, or like at like some of the Chinese teams, and it's just like, I am someone that wants to contribute. I want to do something that, you know, gets a flag that solves a challenge or something. So I can feel good about what I'm doing I can feel like I'm contributing to the team. And a lot of people that come their first time, kind of struggle I think if figuring out what exactly that is, I definitely did, and I think it took me by my second time I kind of felt more like more so like I know exactly like where I fit in best. Um, but yeah I think just having the patience to do that, communicating with your teammates is like super super important, because if you're just sitting there just like not sure what to do and you don't say anything to your teammates, you're gonna be miserable the whole time. And because no one's gonna know that you have no idea what you're doing right. Um, I can definitely say like my teammates, if I ran out of things to do my teammates, I'll just say like hey do you need help and my teammates will be like yeah I could use some help on this or this or this or just like, come join me on the adventure of this challenge or whatnot. And that makes it a lot more fun and I feel like I learned a lot more. So kind of just like you know being patient, communicating and then like humbling yourself like, like, if you don't know something, then you can just sit in with someone else. Put Like you need to tell people that you're struggling and you need to find things to do. And I think those are the most important things for a beginner.
Vamosi: And has any of this CTF experience helped Zaratec in the real world?
Zaratec: I'll talk a bit about, I guess the security community in general things that I see in the security community, things like, you know, bug bounties that people have done exploits that people have found on the news on Twitter on whatever those things when I go and I like open I read about what people did. I'm just like, Man, that sounds like a CTF problem right because you know a lot of the things that we do a strongly apply to what people do in real life. Um, web exploitation, I think is huge, especially for bug bounties because a lot of bug bounties are usually like web applications and, you know, they're very difficult you don't have an end goal but if you have enough practice with the web exploitation and CTF so you might be able to find certain things or apply, you know certain types of vulnerabilities that you found in CTF to that problem and find bugs and get paid for them. Right. Um, and then just other things like you know, crazy things that you see in the news that maybe aren't necessarily bug bounties but you know some hacker, like broke this or took advantage of that. As a CTF or you might be like, oh that was like some type of vulnerability or technique that I saw in the CTF. And a lot of those like new vulnerabilities that researchers find get reapplied back to CTF so a lot of the really difficult ETFs will actually use zero days or one days that are found by these researchers to create CTF problems, and that's how they make it really hard and because they're basically saying here is literally this real world thing, and I want you to find a vulnerability in it. And that's how they make it really hard. But yeah, is CTF so very important to real life work? I think real life work is also very important. Back to get better at CTS as well there's kind of like you need to get good at both and by hopping between them. You can improve yourself above them.
Vamosi: Special shout out to Zaratec for sharing her experience with the world’s greatest hacking team. And not only are they competing at DEF CON but they’re also competing in HackASat - a global competition to hack into a satellite in orbit. This is definitely something we’ll cover in another episode. Until then, I remain, still in sheltering in place, as Robert Vamosi for The Hacker Mind.