Penetration Testing and Ethical Hacking with Ed Skoudis
Is there such a thing as an ethical hacker? Do all hackers use their skills to attack infrastructures? If you’re interested in ethical hacking and penetration testing, this is the episode for you.
Today’s guest is Ed Skoudis. Ed has taught upwards of 20,000 security professionals globally and his contributions to information security have had an immense impact on the community. His courses distill the essence of real world frontline case studies he accumulates, because he is consistently one of the first authorities brought in to provide post-attack analysis on major breeches. He’s not just an expert in the field, he’s created many of the founding methodologies empowered by governments and organizations around the world to test and secure their infrastructures.
Ed is the founder of the SANS Penetration Testing Curriculum and Counter Hack; leads the team that builds NetWars, Holiday Hack, and CyberCity; and serves on the Board of Directors for the SANS Technology Institute. A consummate presenter, Ed is a keynote speaker appearing internationally at conferences, and is an Advisory Board member for RSA.
- [1:14] – Ed introduces himself as a penetration tester and hacker. He does expert witness work on large-scale breaches and incident response.
- [1:37] – He started this path as hacking for phone companies. He explains how he started and why he was hired.
- [3:55] – Ed describes what penetration testing (or pen testing) is. It is when he models the techniques used by real-world attackers and then applies them in a structured fashion to help protect companies from future attacks.
- [4:53] – Pen tests can be used by an organization for a specific reason or can be done as a “check-up” to make sure everything is okay.
- [5:40] – Zero Day is researching vulnerabilities that aren’t known yet. It is called Zero Day because it has been known for zero days.
- [6:18] – Ed never believed the cybersecurity industry would be as huge as it is today and explains some of the more recent issues we’ve been seeing through Covid.
- [7:52] – Security is now a part of the process and is becoming less of an afterthought. Ed shares this idea long-term but sees the same vulnerabilities repeatedly.
- [8:49] – There is no such thing as 100% safe and secure. The goal is to raise the bar to make things more difficult for an attacker.
- [9:31] – With ransomware, attackers have figured out a really reliable way to get paid for their malware.
- [10:08] – Ransomware is a real problem and has even attacked hospitals and local governments.
- [12:37] – There is so much that we do that leaves us vulnerable.
- [13:29] – One major piece of advice that Ed gives in regards to general consumer security is to keep all of your devices patched and updated.
- [14:18] – Another area to be aware of is always spear phishing. Don’t click unless you are confident in the link.
- [15:19] – Ed has a separate computer that is separate and independent that he only uses for financial transactions.
- [17:17] – Chris and Ed discuss routers that consumers buy from their ISP.
- [19:01] – Ed shares how “Live Off the Land” attacks work by using what is already installed on a computer. These are harder to detect.
- [20:18] – With ransomware, organizations have to think about what is best for the business. Is it better to give the attacker the money or spend the time and money to fix the problem?
- [22:27] – The dominant trend in technology today is cloud storage.
- [24:18] – Ed describes how this works and some of the various problems associated with it.
- [25:39] – The US Army, Tesla, and Uber are a few organizations that have lost data to this common vulnerability.
- [26:10] – Ed describes the Holiday Hack Challenge and the fun way he has modeled this problem to educate users.
- [28:03] – Although there are some security risks if not managed properly, Ed shares that cloud data storage is a very cost-effective option for small to medium-sized businesses.
- [29:24] – Most organizations use multiple cloud services. Cloud migration is good if a different service is necessary, but the previous cloud service is left alone and vulnerable.
- [31:01] – The Holiday Hack Challenge is a free educational event that Ed and his team have created to solve cybersecurity challenges. This is a worldwide event.
- [34:02] – The Holiday Hack Challenge is something that some people participate in with their kids due to the video game aspect. There is also a social aspect to it with the chat feature.
- [35:29] – Not only is the Holiday Hack Challenge free, but all of the past challenges are free and accessible to play. This can build your skills.
- [36:30] – Chris asks if Holiday Hack Challenge has ever been compromised. Ed admits that there are three people who have hacked their way in as players, but there were no purposeful attacks.
- [39:43] – Ed admits that he sees the world through the eyes of an ethical hacker because he likes to explore and see what’s beyond the edges.
- [40:48] – Chris and Ed discuss Bug Bounty Programs and can be useful.
Thanks for joining us on Easy Prey. Be sure to subscribe to our podcast on iTunes and leave a nice review.
Links and Resources:
- Podcast Web Page
- Facebook Page
- Easy Prey on Instagram
- Easy Prey on Twitter
- Easy Prey on LinkedIn
- Easy Prey on YouTube
- Easy Prey on Pinterest
- SANS Web Page
- Counter Hack Web Page
- Ed Skoudis on Twitter
- 2020 Holiday Hack Challenge
Can you give me a little more of a background on yourself?
Sure. I am a penetration tester. I’m a hacker. I do expert witness work on large-scale breaches. I do some incident responses. I started doing this stuff about 25 years ago. I was working for the phone companies back then. They gave me a job saying, “Do you want to hack the phone companies?” I said, “Well, yes I do.” We had a lot of fun, and what we found is to do the best hacking we could, we would do incident response. We could see how the phone companies got hacked, learn from that, and then see if we can apply those same techniques to other phone companies in other areas. It’s the virtuous cycle.
In fact, my whole career has been that. It’s learning from bridges so that we can do better pen tests, learning from pen tests so we know to look for the bridges and just kind of flow that way. In the last 25 years, a lot of things have changed, a lot of things stayed the same. It’s been exciting. I’ll tell you, Chris, when I first started my career, I was working in the phone company. My officemate there said to me, “We know how to do good security. You probably have five or 10 years of this cybersecurity crew.” We didn’t even call it cyber in those days. It was information security. “You’ve got five or 10 years, then you better plan another job because we’re going to fix all this stuff.” I’m 25 years into my 10-year career and I’m loving it.
It’s not getting better. It seems like it’s getting worse.
Yeah. I’ve heard it said we’re deploying vulnerabilities faster than we’re fixing them. It’s like a glacier that’s floating in the sea and you see 10% above and then all the ice under the water, that’s the 90%, and we’re adding more ice under the water all the time.
Some of it I think it’s just the longevity of technology. I know when I had worked for a company, I inherited an IT department and I immediately started looking at all of the stuff that me and my current team had. We were conscious about security and we’re slowly working backward to everything that looked like it was still active. We hired a company to come and do some pen-testing. They found vulnerabilities of stuff that was 10 years before my time that was “How in the world did they even find that? It was buried down six folders deep in something we haven’t touched.”
Legacy systems always bite you in the butt. I heard this saying once and I thought it was kind of a funny one. It says, “God created the world in six days.” “Well, yeah, he didn’t have to deal with legacy systems.”
That is too funny. Can you give me a little background on what pen testing is?
Penetration testing is where we model the techniques used by real-world threat actors, real-world attackers. We apply them in a structured fashion to our customers’ networks. Usually, we deal with pretty big businesses, although sometimes small- and medium-sized businesses. We do government institutions. We do some military stuff. We’re using offensive techniques against them with the goal of helping them better manage their risk and improve their security stance. That’s what pen testing is. It’s modeling attacks so that we can make the world a safer and more secure place and I love it. I love it.
Are you primarily going after known publicly facing surfaces or are you looking for anything that you can possibly find, phone systems and WiFi-enabled printers?
Yes, a WiFi-enabled printer is a big one, as long as you get close enough for the WiFi. Usually, there’s a defined scope. Do you want to look at this web application or maybe you deployed a new app that’s tablet-based or phone-based or some new infrastructure an organization has deployed? There’s also just the regular routine, annual pen test. Give us a check-up. You go to the doctor, “How am I doing?” We do that a lot.
We’re scanning and we’re looking for—just like attackers do—clear, obvious—in retrospect—ways in. The attackers usually are focused on the low-hanging fruit. “Let me find that old Windows system that everybody forgot about.” We also do some work, though, that’s pure-play research. Nobody knows this vulnerability is there yet and we find them. This is the realm that they call zero-day because the vulnerability has been known for zero days. We do some zero-day research and find some flaws. Of course, you want to do responsible disclosure. You don’t want to just unleash these things out on the world.
The neat thing about penetration testing is there are so many different types of it. You can do the network pen-testing. You can do web applications. You can do mobile devices. You can do the internet of things—we’ve done a lot with that. If any of your listeners are looking to get into a cybersecurity field, pen testing is a great field. I am biased, I’ve been doing it for 25 years. It grows bigger and bigger. I never expected this, Chris. As I said, I thought I would get 10 years out of this career, and then I’ll be a mid-level IT person in a bank or something.
If you would’ve told me back 25 years ago that pen testing would become what it is or even the cybersecurity industry would become what it is, I would not have believed it. It’s an industry that keeps growing and growing. The whole COVID thing has actually forced people to double down on securing the infrastructure because everybody works remotely. Ransomware is kicking everybody’s butt. They’re calling us in pen testing and they’re saying, “We want a pen test so we can see how ransomware is going to get in.” Ransomware is becoming the justification for pen-testing.
It is crazy times. It’s also a frustrating and sad thing because it’s the reason our industry is doing so well is that everybody else is getting hacked and a lot of nasty things are happening. I have a friend who owns a liquor store and he tells me because of COVID, business is booming. He’s a little sad about that. I mean look, he’s happy having money, but you know. It’s sort of a double-edged sword thing. I’m happy and I’m thankful that business is well and my friend’s businesses are all doing well, but I do wish the world was a more safe and secure place. I wish there was less need for us.
Hopefully, over time as legacy systems go offline, I think security is now becoming a thought process. It’s not just, “If we have money, we’ll throw security on as an add-on.” I think it’s really been baked into the process—not always well, but at least companies are starting to try to bake it into the process and assuming when you build stuff, people are going to try to hack this. People are going to try to break it.
I like that. I appreciate your optimism on that front. I do think I share it very long-term. Maybe it’s because I’ve gotten jaded over the last 20 years in this idea that we’re deploying vulnerabilities faster than defenses or fixing the flaws. It just seems like the same kinds of hacks, the same kinds of vulnerabilities keep getting discovered again and again. We are baking it in more. There’s no such thing as 100% secure. I mean just take your computer, put it in water, bury the all-wet computer, then maybe you’ve got security. Don’t connect to a network. There’s no such thing as 100% security.
There’s this idea that we want to raise the bar so that the attacker has to work harder or even spend more money, and time is money, so spend more time and money that it becomes impractical for all but the richest attackers, maybe nations, states or something like that. That’s where I see the avenue to get to a more safe, secure world is to harden things to the point—not that they’re infinitely secure or 100% secure—but to make it so that routine attacks just don’t work anymore and only really rich attackers can get it.
It’s kind of the concept of good-enough security.
That’s right because good enough is good enough and anything that you do beyond good enough is spending too much extra money, but it does bring us back to ransomware. With ransomware, the attackers have figured out a way to get paid for their malware, a really reliable way to get paid for their malware. Some of that payment goes into R&D. The bad guys own R&D so that they can make nastier malware. I said you know we need to raise our security so that we can require that the attacker spend so much more money, they can’t get over us, but the attackers’ budgets are going up. Why? Because of the success of ransomware.
That’s another ugly trend. I’m sorry, I want to talk about optimism and happy thoughts, but it’s an ugly trend in our industry. This ransomware thing is no good for anybody. It’s hitting hospitals. It’s hitting our local governments a lot. City and sometimes even state governments are getting whacked. The attackers are doing it because they know they’re fairly reliable payers in this. There are law firms now who will take on behalf of a company affected with ransomware, they’ll do the negotiation for you. They’re experts; they’re really good.
My friends and I were joking about how they should set up a reputation service for the ransomware attackers. If you’re negotiating with a ransomware attacker, you want to know if you offer them a certain amount of money, they’re actually going to give you the decryption keys, so you can get your data back. You want to have honest to-their-word criminals. Do you get four stars or five stars for having delivered the key to decrypt the thing after you’ve criminally encrypted it? It’s quite a world.
I’ve always wondered that. I’ve gotten a few. I guess my websites have gotten DDoS threats over the years of, “We’re going to unleash a terabyte DDoS on you. Hope you like your data center.” It’s like, “Give us one Bitcoin, or whatever it was, or $1000.” I always wondered like, “What do I trust more? The fact that you are a criminal or the fact that you’re honest? If I paid you once, what’s to say that you’re not going to go now that I know you’d do this, I’ll threaten you quarterly? Once I know you paid quarterly, I’ll threaten you monthly.”
That’s right. This money gets spent. Yeah, so you’re labeling yourself as somebody who will pay the ransom.
I mean it’s almost like when you answer the phone call and engage with a telephone scammer. Once you’ve fallen victim, you’re now on the hot list of, “Let’s go after this first-person, because they’re going to pay.”
It’s true, it’s true. And they target old people in those things because they’re more likely to do that. They are more likely to have the money, which is another kind of sad thing. In looking at some of the profiles of people who are doing this, they come often from outside of the US. A lot of Eastern Europe, a lot of Russian folks, a lot of folks from Africa and various other countries. Their targets are rich Americans and Europeans, so they already have the money anyway. Why bother caring, which is sad too. Especially when you see sometimes these older people, elderly, getting hit pretty hard and they don’t have a lot of extra funds.
That’s the whole behind-the-scenes of why I started the podcast in the beginning. I’m trying to get way beyond just tech. It’s that same thing. There are so many things that we do that leave us vulnerable. There’s so much low-hanging fruit in our lives that are the biggest targets.
I really like what you do because it’s very practical, very approachable, very listenable. I do like the podcast. You do a great job.
Thank you, I super appreciate that. You talked about targeting low-hanging fruit. For consumers, and, let’s say, small businesses. Let’s throw out the Fortune 500, let’s throw out the people to pay your bills, what are the low-hanging fruits that individuals and small businesses should be looking at? What’s the easy in?
There are two things I’d like to direct your attention to. One is keeping the systems patched. When you get an update notification, a legitimate one, from Microsoft or if you have a Mac, Apple or something for your phone, you have to apply that update. Now, there are illegitimate ones, and one of the biggest ones out there—I’m sure you’ve seen it, everybody sees it. You go to some website. It says, “Your Adobe Flash is out-of-date.”
Close your browser, you’re done. Maybe your Adobe Flash is out-of-date. That’s fine. Update that via some separate mechanism, but don’t click on the thing in your browser that says, “Update my Adobe Flash.” It’s one of the most common pieces of malware out there. Keeping your operating system patched. Keeping your browser patched, so when Chrome tells you it needs to patch, you really need to do that. That’s one.
The second thing, and I’m sure many of your listeners are savvy—they know this—it’s spear phishing. It’s people sending an email trying to get you to click on that link and then hacking your computer if you click on that link. Now, you need to be careful. You need to exercise your due diligence and don’t click on just any link that comes. Double-check the email address that it comes from because some of them just look like nonsense. Now the thing of it is, for simplicity’s sake, if you look at your mail reader on your phone or on your computer, they often don’t show the full email address unless you actually click on it.
I’ll tell you something that every cybersecurity professional knows but most won’t say. All of us eventually will accidentally click on something we shouldn’t click on. It happens. An email comes in, you’re like, “This is my good friend. I shouldn’t have clicked on that one.” Every one of us, every once in a while, we’ll click on a link we shouldn’t click on. You have to make sure you keep yourself patched, because when you click on that link, if you’re not patched, that’s how it’s going to get around it. Keep your operating system up-to-date.
I have another thing for consumers. Now, this costs a little bit of money, but it’s not ostentatious. I have a separate computer, independent from the one that I read email on, independent from the one that I surf the net and watch Netflix or Amazon Prime or whatever. I have this and I use it only, only for my financial transactions online. I never surf the web. I keep it up-to-date, of course. I’ve got to do that. I don’t read emails on it. I don’t do anything other than financial transactions on it. It is such a freeing feeling.
If I were to get some email from my financial institution that comes in my normal email, which I don’t read on there, I then move over to the other machine and I log into that account. I can’t even click on the link in that email because I go to the other machine—having a separate machine. Now, I mean I got an Apple here. You just saw, but you can get a maybe $300, $400 Windows box or something like that. Or for a more tech-savvy, maybe a Linux. It doesn’t have to be a super beefy computer, just a separate little thing on the side that you use just for financial transactions. Not a bad idea. I do recommend that to my consumer friends. I do understand not everybody can afford it.
That’s a good option. One of the things that I’ve seen more and more frequently is people that have a really old…” I moved into my new home and my ISP provided me a $99 D-Link.” Or, “I bought that $69 D-Link router.” Not that I’m picking on D-Link, specifically. Or let’s say MikroTik routers. They were, at some stage, easily compromised, even if you’ve got your computer patched and your OS patched, there’s a little piece of hardware sitting in a closet that you’ve never done a firmware check on and it’s questionable whether the manufacturer even does firmware updates on it.
Exactly. D-Link is an interesting example there because they will often not do firmware updates for stuff that’s more than eight or 10 years old. They just say, “That’s done. We don’t update it anymore and it’s got vulnerabilities.” Don’t even get me started about IOT, the internet of things, updating the firmware of that. I’m surrounded right now by 53 Philips Hue bulbs. I can change their color and all this kind of stuff by talking with my good friend Alexa. I’m not going to say her name out loud. We call her Alice if we don’t want to invoke her. Philips is constantly pushing firmware updates for the bulbs and the hub and keeping that stuff patched is important as well.
When you get to the router, whether it’s a wireless router or maybe a cable modem or file system, because it could be compromised itself, you have to make sure you armor everything else around it. You can’t rely on it to be your sole security. “Wow, there’s a firewall built into my router.” “Good for you, you still need the stuff on the end systems.” Again, patching, and then user awareness, and then a good anti-malware solution. I don’t want to get vendor-specific here, but there’s a lot of decent ones out there. Microsoft’s own built-in one, Windows Defender, has gotten better. Not perfect, but it’s gotten better.
Somebody discovered an issue with Windows Defender not too long ago, where you could use Windows Defender, if you hack into a machine, to download malicious software on the box.
I remember reading about that.
I think they disabled Defender for a couple of days over that, didn’t they?
Yeah. It’s really something. It’s an example of what we call live-off-the-land attacks. The idea is instead of me hacking into your computer and putting malware on there, I’m going to use the underlying operating system against itself. I’m going to do every evil thing I want to do on your computer without any separate software. I’m just going to use the software built-in. I’m living off the land, just like farmers would live off the land as they moved and migrated. I’m going to be a hacker living off the land in your environment. It’s a very interesting attack mechanism and much harder to detect, because there’s no separate malware engine.
The one thing I’ve never quite figured out is with these municipalities and cities, and hospitals—I think it was just recently a hospital chain. I haven’t gone super in-depth on a bunch of them, is the reason they’re paying the ransom is that they didn’t have system backups or because they just thought that that would be a faster, quicker, easier way to get back into service? Just pay the ransomware.
Yeah. It’s a combination of those two. First off, most of the time they have some sort of backup, but if it’s an online backup that is constantly updating itself backing up, the malware will often spread right to the backup system itself. It encrypts the backup. Your production system has been encrypted and so is your backup—that stinks. The second factor is it ultimately becomes a business decision. They’re offering to sell you the decryption key for $1 million and your accountants and everybody else looks at it and says, “Or you could fix it yourself and figure it out or change your business.” “That’ll cost you $3 million. What do you want to do?”
It’s ultimately a business decision on whether to pay. Now, some people actually go beyond the business decision and try to make a moral stance. I do respect that, but a lot of people, it’s just a business decision. “How much will this cost us if we don’t pay it and if we do pay it?” It is faster and may be easier, but you’re identifying yourself as a mark for the next time.
The challenge with the moral stance is, “It’s going to take me longer to get it back and it’s going to cost me more money, can I afford to have my employees not working, my customers.” It becomes not a very simple don’t-ever-negotiate-with-terrorists type of position to…” But I have employees, I’ve got families, I have all these people depend on it.” It becomes a much more murky decision.
That’s right. There was a case that happened recently, I’m sure you saw about it, Chris. It was a hospital—I think it was in Germany, I’m pretty sure it was in Europe—where the hospital shut down, somebody was in grave medical condition being rushed to that hospital in an ambulance and they had to divert because that hospital shut down.
They diverted to a different hospital and the person died on the way there. At least, it’s the first documented death. I won’t say due to ransomware, but in association with it, ransomware contributed to the circumstances that caused that person to die. That sucks.
Plain and simple, it totally does.
One of the things that I consistently see showing up is—in association with data breaches—these unsecured cloud containers. Could you tell the audience, Ed, what the heck is an unsecured cloud container and why is this happening repeatedly?
It’s really something, the cloud. It is the dominant trend in computing today. Rather than having all the computational resources on your desk or in the company’s data center, you have some other very big, very rich, very smart company do that, like Amazon, AWS.
AWS is the biggest cloud provider today. Number two is Microsoft with its Azure infrastructure. Number three is probably Google. Google is not used to being number three in anything.
It’s good for them to know what it’s like to be the child.
They are trying hard. We host things from our environment in all three of those because we need the experience with all of them. It’s a need. It can be very cost-effective and it’s almost magic. You just have these API calls that you make often to the sky and the cloud computes things for you. One of the biggest things that you want from the cloud is the ability to host and process your data.
The whole argument here is rather than buying computers for yourself that you only use a small fraction of their computing power, I’ll have somebody else buy the computers. I’ll still use a small fraction of the computing power and I’ll pay for that fraction. Storage is getting so cheap so I’m only going to use a small fraction of the storage that I need.
One of the big draws to the cloud is putting your data in the cloud. Now, cloud providers give you all kinds of security, access controls, and all the stuff that you could put on there to protect your little piece of data in their bigger data store.
The biggest cloud data store is AWS and they have these things called S3 buckets. You put your data in S3 buckets. Data’s only useful if the people who need it can access it. There are various crypto keys and API keys that they use to access that, but when you’re building a new application that needs to get access to the data and you’re trying to make it work—you’re a software developer and you’re trying to make this thing work with the AWS S3 bucket data store—you write your code and you give it the API keys that it needs. You give it the crypto keys that it needs to get access to the data and it’s just not working.
You try this, you think it’s your code, you change the code, and then you do this. Well, “Let me turn the security off.” After turning the security off, it still doesn’t work. Then, you start making other changes, this and that, and then it starts working. “Dude, it’s working, I can’t believe it.” The application looks sweet, it’s sexy, it’s fantastic, you’re pushing the production, and you forgot to turn security back on.
What happens is these organizations oftentimes have really just disabled security entirely in these cloud-based data storage. They think nobody’s going to find them. They think, “Well, this is buried in these giant AWS infrastructures, confined in this one little S3 bucket, but I have a hundred gigabytes of sensitive data in it.”
But these hackers, they’ll start trolling just for those. They’re going through the environment. They’re looking systematically through all the possible S3 buckets until they find one that doesn’t have security. They look inside and say, “What’s in here?”
If you look, lots of organizations fall into this. The US Army lost about a hundred gigs of data through this attack. Tesla has been hit this way. Uber was hit this way. Several other different organizations have been.
Some of the ISPs are storing data. Interestingly enough, ISPs are storing data in AWS cloud. It’s a very common kind of attack and a common kind of vulnerability.
We modeled one last year in our Holiday Hack Challenge. That’s where we set up this fun hacking challenge and cybersecurity challenge.
It’s based on Santa Claus at the North Pole. Santa Claus had a GitHub repository where the elves were storing some code, and the Git repository was not marked as secure so people could get into the elves’ code. Inside the elves’ code were some keys to then access the cloud-based data store. We were trying to model that to teach people about how bad a problem is.
Is this something that AWS, Microsoft, Google—are they culpable in this, that these environments are being set up as a default without security and the user’s not implementing it, or is this often more on the users’ side where the security’s on by default and they’ve actually turned it off?
Yeah. It used to be the security was off and you had to turn it on. But you could imagine these cloud providers, they don’t like the bad press from this. They’re turning the security on and it often gets turned off during that development process like I just described.
It’s interesting to watch the evolution of Amazon S3 buckets on this in particular because it used to be that if you left the security off and you just looked at the overall dashboard, you could just look at the dashboard to see all of your S3 data stores, and be, “OK, they’re all there, they’re all functional.”
But starting about a year-and-a-half ago, if you have one word, security’s effectively turned off. There’s no access control on it. In the user interface now, in your cloud-based admin port, it lights that sucker up and says, “Look at this one. There’s no security, are you sure?” People still miss it. There’s somewhere you don’t want security because you’re trying to share data publicly and there’s others where you do want security.
If you work for a small business or a medium-sized business, you’ve got to have your admins. Make sure that they are looking at the security, especially the configuration of the data stores.
Let me tell you something else, Chris. I do think the cloud offers small and medium businesses the ability to improve its security in a more cost-effective fashion than they could if they just tried to hire it all on their own because you have the smarts of big, rich companies like AWS, Microsoft, and Google. That’s there, and that’s really cool.
They can look in various services to look for attacks and things like that, but you do have to do a configuration. You’ve got to make sure that you are applying the security capabilities they have, doing some log analysis, and such. There are tools available like CloudTrail and CloudWatch to look through the access of your systems and tell you if they see anomalous things. Microsoft has this similar kind of thing for their databases in the cloud.
That’s good that the platforms are starting to realize, “Oh, maybe the users aren’t as tech-savvy as we had hoped they would be.”
Yeah. It is interesting. Everybody’s trying to learn how to deal with our new cloudified universe.
Another thing I say to a lot of chief information security officers or CISOs is I’ll say, “So which cloud do you use?” They’ll always say, “Well, we use this one.” I’ll say, “Are you sure you just use that one?” “Yeah, we’ve got most of our stuff in AWS.” “So there’s no Google, there’s no Azure?” “Well, there are a couple of projects in that.”
The truth is, everybody says we’ve moved to the cloud when the reality is we’ve moved to the clouds. Most organizations might have 90% or 95% of their stuff in the one cloud and then they have the rest in some other cloud that nobody really thought of. Of course, that’s the one that’s not getting the security scrutiny. It’s the low-hanging fruit strategy we talked about earlier, moved to the cloud.
For most companies, does cloud migration result, on average, better security in that you now have the maintenance of the platform itself handled by someone who’s an expert in that?
Like if you ask me, “Hey, Chris, we need you to maintain this database server securely. I don’t know enough about the vulnerabilities on the box itself.” Sure, I can remember to put patches in, but I’m not going to know the little nuances.
I believe the answer to your question is yes, especially for small and medium businesses. The move to the cloud does improve things for them because the cloud is taking care of a lot of stuff that they don’t have the resources necessary, or the ability to hire the knowledge to do that.
That said, it’s still got to be configured right. If it’s not configured right, you’re going to have an open S3 bucket and get whacked. But I do think it is a measurable improvement in security, this move to the cloud.
I like that. You’ve talked about a Holiday Hack Challenge that you guys did. Can you tell me more about that?
Yeah. It’s so much fun. We’ve been doing it for almost 20 years now. The idea is the second week of December—not too far off—my team releases a cybersecurity set of challenges. We work on it all year round, it takes us more than a year to put one together. What happens is we release it at holidayhackchallenge.com, and it’s totally free. We make no money on it, it’s our gift to the community.
You go to that site, you create an account, and you’re given a little avatar. You can control how your avatar looks. You can choose your head, your eyes, your mouth, your body, and your legs. Once you’ve created your avatar, you walk into this little Christmas-themed world and you see Santa Claus. You can talk with Santa Claus.
Here’s the deal. Somebody, every year, is trying to destroy the holiday season. There’s custom music—we get custom music made for the whole thing—holiday music. You have to solve various cybersecurity challenges: defensive challenges, forensics challenges, offensive challenges. You solve those with the goal of trying to figure out who is trying to destroy Christmas and why, and then you stop them. You thwart their nefarious activities.
The challenges start very simple. We have a lot of kids that play this—junior high kids, high school kids. But we also have some of the most hardened security professionals at the NSA play this.
People play it around the world. Almost every major military service in the US plays the Holiday Hack Challenge. They play throughout Europe. I know the Japanese police force at the country level—they play it a lot. I know there’s lots of fans in Saudi Arabia of the Holiday Hack Challenge. It’s really cool. Australia, there’s a lot of people that play there.
We had 17,000 people that played last year and we’re hoping for even more this year. It’s fun, it’s whimsical, it’s weird. There’s prizes and stuff. We try to make it a good time.
One of my favorite parts of the Holiday Hack is—I’ve had many friends say this to me. They say, when we release it, that second week of December, a lot of cybersecurity professionals are focused on it. They play it. It goes for a whole month. They play it for the whole month. Some of my friends have said to me, “If you’re an evil villain, just a terrible person, cybercriminal, nation state, whatever, and you want to do a horrible, nasty attack, do it on the day we release Holiday Hack because nobody’s watching.”
They’re all helping Santa Claus, so that’s your chance to really do your thing. I actually think that’s wonderful, I love that. I do hope somebody keeps an eye on the actual rest of the world while everybody’s at the North Pole. It’s a fun challenge.
We release it in context with something we call KringleCon. KringleCon is Santa’s virtual conference at the North Pole. When you go in, all through your browser, you go to the North Pole, you hop around Santa’s castle, and you can go and watch videos. The videos are presentations that will help you solve the hacking challenges. That’s an integrated thing.
A lot of people will play with their kids because there’s a video game aspect to it. They’ll have their kids hop around the castle and say, “Oh, I need the firmware for this device.” They have to hop around and find the firmware. It’s in there. Or, “I need you to find what cipher was used for this.” There’s this video gameplay.
There’s a social aspect to it, too, because there’s chat back and forth. Last year, we saw the emergence of Holiday Hack parties where different places—there was one in Australia, one in Frankfurt, Germany, there was one in New York City, there were several others we don’t even know about—where companies hosted a Holiday Hack party at their venue so people could just come in. There’d be like 40, 50, 60 people sitting there and playing Holiday Hack all together at the same time with Christmas music blaring, cookies, and things like that. Now that we live in COVID world, we’ll see how that goes this year—maybe virtual Holiday Hack parties.
Everybody beat the mask, socially-distance, desks six-feet apart.
Yeah. It’s such a different world. Anyway, the Holiday Hack is coming up. I encourage your listeners and viewers to play.
Again, we don’t do that event as a profit motive or anything like that. We do this to serve the community and because it’s so much fun. We really want to help bring up the next generation—I know that’s something that you have a passion for—and to try to make the world a better place.
Here’s another big, important point. People can play previous years’ challenges, they’re all up. I pay many thousands of dollars a year in cloud services to keep them up for everybody, comes right out of my own pocket. If you want to play Holiday Hack Challenge 2018, it’s still there for you. Or 2017, it’s still there. All from 2015 to today, they’re all up all the time, 24 by 7 by 365, and you can learn from them. You can build your skills there.
People will say, “Well, how can I get ready for this one—the one coming up?”
Play last year’s.
Play last year’s or the years before. They’re fun, little stories. All the answers are posted for the earlier ones, so you can see the answers and work your way through them. You’re like, “Well gee, I’d like to have a cyber range to practice my skills. Holiday Hack Challenge is the year-round cyber range that you can build your skills on. For free.
Ed, I have to ask the question. Has the Holiday Hack Challenge ever actually been compromised?
Wow, that is an interesting question. I’ve done a lot of podcast interviews, nobody’s asked me that. We have had, on three occasions off the top of my head, instances where people got into the underground of Holiday Hack Challenge, the back-end infrastructure.
They were all really wonderful people and they responsibly disclosed to us that they were in there. We thanked them so much and we did, during our closing ceremonies, point out that somebody got into this. We asked them if we can announce their names because it’s them.
I’m not trying to lay down the gauntlet. I’m not saying someone should try to hack our infrastructure—it’s against the rules. That said, if they do find these holes, maybe even if they compromised them and they tell us about them in a good spirit, we’re very thankful for that.
There’s always this thing in your gut a little bit that says, “Tell them it was an Easter egg that you’ve purposely built and they found it. Oh, you’re brilliant.” No, we fessed up and usually I get on the phone and I’ll say, “Hey, thanks for finding this, we will announce that you found it if you want us to disclose your name. We’ll announce that it was there, but we want to make sure that you’re dealing on the up and up and all that.”
We don’t have a bug bounty program for it because it’s not for profit,—we make no money on it. Last year, we had these things made up. I told you it’s KringleCon. This is a Kringle coin and we give the Kringle coins to the winners of the challenge.
But also, if somebody finds a vulnerability or an issue, it’s good. The coin costs $7 so it’s not a bug bounty but people love the coin and they want the coin. It’s worth more than $7 but it costs us $7 to have it manufactured.
It’s a brilliant and insightful question. On three occasions, people have been able to compromise it. We did have one last year. It wasn’t really a compromise of the underground part of it but they were able to find an issue, a vulnerability in the above-ground part so they could get around faster and do things that they otherwise couldn’t do. We love that, that’s just awesome, right? But when you get into the underlying data structures and can manipulate the game, or see the information you shouldn’t be able to see, that I call a compromise. There have been three but they’re from very good-willed people.
I guess that’s just the underlying…kind of that good-enough security, so to speak. If it’s connected, there really is no absolute security.
That’s absolutely true. I’m bragging a little bit here, but I’m telling the truth. We have some of the smartest people in the world that are playing this game and there are also little kids that are playing.
There’s usually 10 or 12 challenges and there are two or three that almost anybody can get if they try, but then there are two or three that you have to be about the best person in the world to get this. Sometimes on their way to those last two or three, they notice anomalies and they start exploring them because they think it might be part of the game.
Next thing they know, they’ve blown a hole through the floor of the game, that infrastructure and how everything works, the data flows.
It might be because of my own background as a teenage explorer and hacker. I know you did a lot with the BBS scene back in the day—80s or 90s—I did too. I didn’t run one but I participated in a lot. I was in Michigan, you’re in California, I believe. That was a long distance call, I couldn’t call you.
That said, sometimes you could ride across connections that other people had. I do view the world as a hacker. By hacker, I don’t mean nefarious criminal. I mean somebody who likes to explore, likes to learn, likes to see where the edges of things are, and sees what’s just beyond the edge. I think it’s because of that hacker ethos, I don’t really get mad when somebody breaks that that way, as long as they do it with goodwill. If you do it to be evil, something like that, or you start asking for money, I don’t put you in that hacker mentality. Hacker mentality is, “Let’s learn, explore, and do it to try to make the world a better place.” Ethical acts, as some people call it.
Yeah. I like that. I know we’re tight on time here. Bug bounty programs, are you in favor, not in favor? Do they attract more attention to people who don’t necessarily want attention drawn to themselves?
I’m in favor, let me just say that right upfront. I do think they help. They do draw attention so only get into one if you’re a company and you have a product or infrastructure. Only get into one with your eyes open. If you’re a participant, if you want to look for these flaws, make sure you stay within the scope of the bug bounty rules for that given organization.
I do see bug bounty programs for some of my customers, Fortune 500 companies, that turn stuff up. Either stuff that came up between our pen tests or that we never pen-tested because it’s just a different scope. I do think it augments a security practice, but you’ve got to do it carefully.
There are companies that can help you set up a bug bounty program. Again, we’re vendor-neutral; I don’t have any allegiance or alliance in any of these companies but one of them is HackerOne. There’s a couple of others out there as well.
I think bug bounties have proven themselves far more useful than I thought they would 10 years ago. People come to me and say, “I want to be a pen tester; should I do bug bounty programs to start building my skill on my resume?” I’ll say, “Yes, but realize that most of the easy bugs have already been found.”
You’re probably going to find either hard bugs or nothing. If you’re new at this, unless you’re a super genius, don’t expect that you’re going to get paid by bug bounty programs right away or ever. But it can still help you get that job.
What I’ll tell people is record—maybe with handwriting, or something, on a notebook, or maybe type it into a thing—record your work in the bug bounty program.
Pick your favorite bug bounty program that’s out there. Maybe you’re doing one for Facebook. Facebook’s got a big bug bounty program. You probably won’t find anything. It’s Facebook. You might, but it’s unlikely. Write down what you did, what your observations were, when you did it. Because if I’m going to hire you as a pen tester someday, I need somebody who knows how to put those things through their paces, and who is recording what that person is doing so that we can do reporting and all that kind of stuff going forward.
Always build the habits and skills of a good pen tester. Document that because that’s going to help impress your boss down the road, even if you don’t get paid $40,000 by Mark Zuckerberg for finding something in Facebook.
Zuckerberg has said if you find a big enough flaw in Facebook, he’ll give you $1 million, but that said, he’s got his own people that are looking for stuff and a whole bunch of other people that are looking for stuff there, too. I think they’re valuable for pretty much everybody, but don’t think you’re going to hit the jackpot as a practitioner.
It sounds like it’s good to build your methodology, to work on your craft, but the chances of being a professional bug bounty hunter is probably not going to pay the bills.
That’s exactly right. I do have a couple of friends who do this and they make some pretty good money on it. Kudos to them but they’re very much the exception, and they’ve been doing it for a decade or more. A newbie shouldn’t expect to find anything, but they should still do it if they want a nice, systematic way to build their skills.
And then responsible disclosure.
Yes, exactly. Well said.
Ed, as we wrap up here, if people want to get ahold of you and find out more about what you do, how can they find you?
Sure. My twitter handle is @edskoudis. I’m an instructor with the SANS Institute—that’s where I teach people how to do penetration testing, so they can find more information about me over at sans.org.
My company that actually does the penetration testing is called Counter Hack, so if you go to counterhack.com, they can see more about that. Our company, if you look at Counter Hack, we were founded 10 years ago. We just celebrated our 10th anniversary. We’re founded to do really high-quality work with people that we love to try to make the world a better and safer place.
That’s why we do the SANS Holiday Hack Challenge. My team works on that, funded by the SANS Institute, to try to make the world a better place. That’s ultimately about the community. At the end of the day, you’ve still got to feed your family and I’ve got to pay my team, but we really do want to try to make the community a better and safer place.
I love it. Ed, thank you so much for coming on the Easy Prey Podcast.
Thanks, Chris. It’s great talking with you. Thank you, buddy.
- Easy Prey Podcast
- General Topics
- Home Computing
- IP Addresses
- Online Privacy
- Online Safety
We all have limited resources when it comes to time, money, and energy so in a world…[Read More]
Social media is a part of life. Given the state of the world, social media has become,…[Read More]
The way we vote is changing – and not for the better. Voting practices in the United…[Read More]