DEFCON 16: Nmap: Scanning the Internet

Summary

In this engaging talk, Fodor from Insecure.org and the Nmap project recounts his experiences and insights into the world of internet scanning, particularly focusing on the tool Nmap. This presentation at DEFCON 16 explores the process of scanning tens of millions of hosts, highlighting challenges such as identifying effective scan techniques and managing legal concerns. He shares strategies for improving Nmap’s efficiency, discusses new features, and advises on optimizing port scanning techniques—providing both technical and humorous insights along the way.

Highlights

• Fodor's adamant stance on the focus of the talk being on port scanning, despite other tech trends. 🎯
• The intriguing journey of scanning millions of internet hosts using Nmap, overcoming technical and legal hurdles. 🌎
• Humor in handling ISP concerns, with a comic take on using a neighbor's Wi-Fi for hiding scans. 😂
• Challenges with large-scale scans, including speed and legal issues, were cleverly navigated. 🚀
• Breakdown of technical processes like host discovery and the significance of effective port selection. 🔍

Key Takeaways

• Port scanning is the core focus, with a caution that those uninterested might find the session daunting. 🌐
• Collecting empirical data through internet scanning leads to valuable enhancements in Nmap's features. 📈
• Innovative methods were used to minimize legal risks and irritations from ISPs, showcasing humor and complexity in handling network security ethics. 🚓
• A variety of scanning methodologies were discussed to maximize efficiency, including host discovery techniques and penetration testing strategies. 🕵️‍♂️
• The importance of upgrading Nmap for better performance and access to newer features was emphasized. 🔄

Overview

At DEFCON 16, Fodor from insecure.org and the mastermind behind Nmap, captivates his audience with the intricate art of internet scanning. His presentation is a rollercoaster through the world of port scanning — a field brimming with technical challenges and legal hurdles. Fodor's humorous and engaging approach makes the complex subject more relatable, appealing to both tech enthusiasts and professionals alike.

The presentation dives deep into the technicalities of using Nmap for scanning purposes, emphasizing the necessity for empirical data collection. Fodor elaborates on how his summer project of scanning tens of millions of hosts could contribute to enhancing Nmap’s efficiency and reliability. The session provided not just theoretical knowledge but also practical insights into conducting large-scale scans efficiently.

Fodor's session is rich with anecdotes and practical advice, such as strategies to overcome ISP scrutiny and legal heat, emphasizing ethical practices in network security. His lighthearted narrative on using one’s neighbor’s network (hypothetically) adds humor to the otherwise tech-heavy discourse. This talk is not only informative but also reminds tech aficionados of the joy and thrill of network scanning.

Chapters

• 00:00 - 01:00: Introduction and Speaker Background In this opening chapter, Fodor from insecure.org and the mmap project introduces himself and expresses gratitude to Defcon for the opportunity to speak at the conference. Fodor emphasizes the significance of community-driven events like Defcon that cater to technology enthusiasts and hobbyists, who might not have corporate backing. Defcon, being an inclusive space, attracts attendees through its affordable admission, questioned on the form about one's willingness to speak, highlighting its community-centric approach.
• 01:00 - 02:30: Purpose of the Scans The speaker expresses their disinterest in attending the event contingent on acceptance by "black hat" and emphasizes their intent to present at "def con" regardless. They clarify that their talk will not cover topics such as cross-site scripting attacks on social networks or hijacking Twitter feeds.
• 02:30 - 04:30: Challenges in Conducting Scans The chapter opens with a discussion on the focus of the session, which is predominantly on port scanning. The speaker makes it clear that the majority of the session will be centered around this topic, humorously suggesting that anyone not interested in port scanning might find it tedious. Comparisons are drawn with Dan Kaminsky, a notable figure in the field, and his frequent discussions on DNS. The speaker indicates that while the primary subject is port scanning, there might be brief excursions into topics like OS detection and the nmap scripting engine, similar to how Kaminsky often ties DNS into diverse topics.
• 04:30 - 07:00: Scan Implementation and Legal Issues The chapter titled 'Scan Implementation and Legal Issues' discusses the implementation of internet scanning and the potential legal issues involved. The author shares personal experiences, highlighting a summer spent scanning tens of millions of hosts on the internet and collecting data. While people often question the purpose of such scanning, the author believes that scanning is valuable for its own sake.
• 07:00 - 10:00: Firewall and Performance Challenges The chapter "Firewall and Performance Challenges" discusses the speaker's goals in collecting empirical data to enhance end map and add new features. The speaker aims to demonstrate how knowledgeable individuals can use this data to improve the effectiveness of network scans, highlighting the common assumptions people have about network structure and population.
• 10:00 - 12:00: Host Discovery Techniques The chapter 'Host Discovery Techniques' discusses the importance of using empirical data to optimize scanning techniques in network security. It notes that assumptions about network setups are often based on personal experience and might not accurately reflect other networks. Therefore, finding empirical data that matches your specific needs is suggested as the best practice. If such data is unavailable, the chapter aims to guide readers on how to conduct scans to collect necessary information. Additionally, it touches on detecting and resolving bugs and performance issues in the nmap scanning tool.
• 12:00 - 15:00: Upgrade Nmap and Top Ports Feature The chapter discusses the challenges and solutions encountered when scanning tens of millions of hosts using Nmap. It highlights the exposure to various networking situations and the subsequent improvement process. Key issues addressed include fixing crash and deadlock bugs, as well as optimizing the speed for better performance. The chapter also aims to demonstrate new techniques.
• 15:00 - 18:00: Scan Speeds and Practical Tips The chapter discusses the practicality of scan speeds and techniques for conducting effective network scans. It emphasizes that if a scanning method is successful for a large number of hosts, such as 25 million, it is likely to be effective for smaller numbers, like 25,000 hosts. The chapter highlights the challenges associated with launching large-scale scans and suggests conducting numerous smaller, yet substantial, targeted scans instead of one large scan. This approach aids in managing resources and optimizing scanning efficiency.
• 18:00 - 23:00: Nmap Scripting Engine and New Features The chapter discusses the challenges of determining which IP addresses to scan using Nmap and mentions various options for selecting target networks. It highlights the usefulness of the Nmap Scripting Engine in achieving this goal. The chapter also describes different methods of finding routable networks, including using BGP, DNS zone files, and registry allocations from organizations like ARIN and RIPE. Ultimately, the chapter explains the decision to use Nmap's random IP generation feature to streamline the scanning process.
• 23:00 - 26:00: Nmap Contributions and Community In the chapter titled 'Nmap Contributions and Community,' the focus is on managing large numbers of IP addresses for scanning using Nmap, a popular network scanning tool. The speaker describes a method for handling a substantial quantity of IPs by generating 2,200,000 addresses, intentionally adding an extra 200,000 to accommodate potential duplicates. Instead of performing an immediate scan, they mention executing a list scan to compile the addresses first, allowing them to plan subsequent scans efficiently. Key techniques include avoiding reverse DNS lookups to save time and using command-line tools like grep, O, and sort to filter and sort the IP addresses. By removing duplicates and selecting a manageable portion of the data, they prepare a refined list of 25 million IPs for further processing. This method highlights a systematic approach to IP management within the Nmap community.
• 26:00 - 27:00: Q&A Session This chapter outlines a Q&A session focusing on cybersecurity topics, specifically on scanning IP addresses. The main discussion revolves around the generation of random numbers for the IP lists necessary for scanning. Once targets are chosen, the conversation shifts to determining what sources to use for scans. Various ideas are discussed, ranging from conventional methods to more unconventional ones such as peer-to-peer (P2P) scanning. This would involve distributing a client called 'n mapster' for users to download. This chapter offers insights into innovative and traditional approaches to cybersecurity scanning.

DEFCON 16: Nmap: Scanning the Internet Transcription

• 00:00 - 00:30 uh my name is Fodor uh from insecure. org and the mmap project I'd like to thank you all for coming and of course Defcon for inviting me I'm a big supporter of community conferences like Defcon where people can go if they have a passion for the technology amateur hobbyists who might really love this stuff but don't have a company that uh will pay thousands of dollars for a ticket uh for example the uh Defcon admissions uh form has a question saying hey is your willingness to speak
• 00:30 - 01:00 contingent on whether black hat accepts you and I was like hell no this talk was prepared for def con and if anything the contingencies are the other way around so uh thanks anyway I'm very glad to be here and I first want to warn all of you that this talk is not about cross-site scripting attacks on social networks or uh hijacking Twitter feeds or anything
• 01:00 - 01:30 like that it's about Port scanning and more Port scanning and if you don't like Port scanning the next 50 minutes are going to be your worst nightmare because for me to talk about something else would be sort of like Dan Kaminsky doing a talk which doesn't involve DNS in some way I mean sure I may throw in some OS detection or nmap scripting engine action just as Dan may take his DNS use it to Tunnel
• 01:30 - 02:00 YouTube in order to Rick Roll some poor schmuck um but in both cases uh we're just expanding on our core topics and my topic as you can see from the title slide is about internet scanning I spent a lot of time this summer scanning tens of millions of hosts on the internet uh collecting data and when I tell people that they're often like why and to me I think scanning is its own reward and you don't really need any
• 02:00 - 02:30 particular reason but in this case I did have uh some concrete goals for the project uh one of them is collecting empirical data that I can use to enhance end map and add cool new features and I'm going to talk about some of those features in this presentation uh second is I want to show how you can use that data for knowledgeable people to make your scans more effective I basically there are a lot of people who make assumptions on how Network are structured and populated and
• 02:30 - 03:00 use those to decide what sort of scans are going to work best uh but these assumptions are often based on hey how would I set it up and they're not always reflective of other networks so when you can find empirical data that meets what you need then that often works best and if you can't find the data a goal of this talk is to help show you how you can do scans like this and potentially collect it I also wanted to detect and resolve nmap bugs and performance issues
• 03:00 - 03:30 the idea is that when you scan tens of millions of hosts you're basically putting nmap through a lot of different situations pretty much any networking situation you can imagine and see how map reacts to it and you know I fixed a crash bug I fixed a deadlock bug uh there were a number of cases where I was like this is going too slow there's got to be a way to speed it up and so then I uh look through and try and figure out why it's slow and improve that I also want to demon rate techniques that can
• 03:30 - 04:00 be good for routine scans as well as the wides scale scanning that you may do the idea is that if a scan works well for 25 million hosts then surely it'll probably work well for you for just a 25,000 or whatever you might be doing now let's look at the challenges to launching such a scan uh first of all I want to mention that instead of doing one humongous scan I did a lot of uh smaller but still large targeted scans
• 04:00 - 04:30 Each of which were designed to you know collect a certain piece of data which would be useful and the question is how should you figure out what IPS to scan I have a lot of options you could take bgp and look at what net block networks are routable and use those uh DNS Zone files registry allocations like aarin and ripe uh but in the end I decided to use nmap's own random IP generation script uh feature uh here we take end map we
• 04:30 - 05:00 say generate 2,200,000 IPS and in this case I did the extra 200,000 because of potential duplicates uh we say do a list scan so don't actually scan the machines just list them out for me because I'll scan them later uh hyphen n you know don't do reverse DNS because that would take a long time and we don't need the data I use grep and O to grab the IPS I sort them remove the duplicates grab the first 25 million and then I have a 25
• 05:00 - 05:30 million IP list that I can use uh for the scans so that's sort of the type of uh way I use generated the random numbers but once you have what targets you want to scan the next question is what sort of source you're going to use and here I had a lot of ideas some crazier than others the first one was uh P2P scanning I was going to distribute a client which would be called n mapster and people would uh download it
• 05:30 - 06:00 and it would scan for them and let them know and re upload the results uh for collection um but I decided that a key goal of this was to make mmap faster and more efficient for people's normal day-to-day scans and so I decided it was better to focus on just using nmap itself rather than building custom software uh for this project that may get around performance issues and the like another big concern was a legal one I knew that when you're scanning this many hosts it's going to raise a few ey eyebrows and I certainly don't want to
• 06:00 - 06:30 get kicked off my ISP again and uh you know being arrested would be much worse so I thought how can I do this but not collect too much heat and so the solution I decided on was to go through my neighbor's open wireless access [Applause] point no I'm uh I'm just kidding about that I decided it would be completely unethical and inappropriate and she didn't have enough bandwidth to make it
• 06:30 - 07:00 reliable so I uh took uh I decided to use an ISP I use for a collocation and do the scans from there I thought maybe it'd go under the radar but it didn't within like 15 minutes of the first Scan they were contacting me frantically saying what the hell are you up to they thought maybe I was infected by one of the most virulent internet worms they'd ever seen they said your machine is going crazy it's probing thousands of machines per second all
• 07:00 - 07:30 over the Internet they were talking about shutting me down and I was like oh no this is uh this is no good um and but then I said hey you know don't worry I'm not affected I'm doing this on purpose and that that didn't help either my case at all basically they figured I must be some sort of spammer or worse if that's even possible um so uh so then I was like uhoh I'm totally busted I'm going to have to cancel the project stop
• 07:30 - 08:00 the scans write a whole new talk on the uh crossy scripting vulnerabilities and give that instead uh fortunately though it turned out that they were end map users so I said hey the scan is to make end map more efficient and effective and they're like oh then carry on so I was uh real happy about that I had to slow it down quite a bit so it didn't melt their switches anymore but uh other than that they were cool unfortunately the US Department of
• 08:00 - 08:30 Defense was not quite so accommodating they uh didn't like my scans at all they said hey you're scanning sensitive military installations this has got to stop and I thought hey you know I'd be happy to use n Maps exclude file option to uh skip those networks um but uh they wouldn't even give me the networks because that's sensitive military information too so uh whatever
• 08:30 - 09:00 um the next issue I mean it has been making me a little nervous now they are the military when planes fly overhead but uh nothing nothing too bad so far the next issue were firewalls uh for many of these scans just pure internet results was all I needed uh but for other ones it would be nice to get a view behind company's firewalls and because they often have you know different ports open the network looks a lot different behind there and I'm happy to say that I was able to get through a
• 09:00 - 09:30 number of these firewalls not through some sort of advanced fragmentation attack but I used a technique uh known as asking them for the data which uh works pretty well uh at least with some of them but there are a lot of big companies who scan their networks every day uh with nmap and we're happy to contribute some data to make it work better another challenge is performance and accuracy and this was different than uh many other types of the challenges
• 09:30 - 10:00 because I wasn't trying to find a quick hack workaround type of thing instead this was a key goal was to improve end map's performance and so I took this as a challenge to see where I could improve but even so it could be disheartening at times like I did a UDP scan with 65,000 ports and I told it to scan 248 hosts in a group and you can see here it's taken 4 days it's on the 1 2048 and I have negative 688 hours
• 10:00 - 10:30 remaining when your time estimate leads to integer overflow and goes negative that's never an encouraging sign uh this particular scan is still running right now and maybe for a while Defcon 2009 I'll let you know how that one's going but fortunately some of our other scans uh finished a lot earlier so that's sort of the introduction to the different scans we were doing and why so now let's get to
• 10:30 - 11:00 some more practical advice that can be concrete details and let you know how you can use this to help your scans and a good first place to start is host Discovery because the first thing you want to do in network reconnaissance generally is host Discovery where you scan the network and try and figure out which hosts are actually available on the internet on the network so that you don't waste a huge amount of time you know scanning IPS that have no host listening on them at all uh but it challenge there is deciding what methods
• 11:00 - 11:30 to use for Discovery there was a time when pretty much all the hosts would respond to an icmp echo request or ping packet unfortunately that time was a decade ago so now a lot more companies uh block those ping packets and you need something more effective Now nmap by default will also send an a packet to Port 80 which helps in eliciting responses but even with that I don't think it's comprehensive enough if
• 11:30 - 12:00 you're scanning the internet or even some sorts of internal scans uh so let's look at some of the different methods that you can use uh TCP we have two types of probes we have the sin probe and we have the AC probe and they're both useful against different types of firewalls the sin probe is likely to get through stateful firewalls when they're configured to allow incoming connections because they'll hey say hey this is is a sin packet it's initiating the connection
• 12:00 - 12:30 let it through but those same firewalls if you send an act packet they say hey this doesn't correspond to any existing connection it's not acknowledging any legitimate data so they'll just drop it and it'll be ineffective however against stat list firewalls you have the opposite problem they may try to block incoming packets to certain ports and they look at the sin uh flag to detect that it's an incoming packet and they'll drop it however you send the ACT pack it against those ones and they have no way to know
• 12:30 - 13:00 they have no state to say whether there's an established connection there or not so they have to let it through so I have a quick example we can do basically let's say we're going to do nmap do a ping scan we're going to do a sin probe to Port 80 no reverse DNS and this time we'll just use send.com and it responds pretty quickly and says the host is up so we got a
• 13:00 - 13:30 synac or a reset back now we'll do the same one with the Knack probe against the same host and it takes a bit longer and eventually it times out and says no response was received and so you want to think to yourself hey is their firewall a state full firewall or is it a stateless firewall and if you think about it for a minute you'll hopefully come to the conclusion that it's a state full firewall because it was allow it allowed the sin package in but the ACT
• 13:30 - 14:00 packet it was able to detect hey that's bogus I'm going to block that sucker so now let's say we've seen the sin probes and the ACT probes and the question is which one do you want to use because the sin probes work against some hosts with the state full firewalls but the ACT work against others and the answer is that hey this is not an either or situation you should use both probes against various ports to have a maximum
• 14:00 - 14:30 chance that at least one of them will get through and generate a response that proves that the host is online so then the next question is what ports should I use you have 65,000 options there and you often don't know which ones are going to work best so here I did some of that empirical data stuff and I scanned hundreds of thousands of machines and I detected the ones that had a heavy firewall the ones that blocked the vast majority of the ports because the ones
• 14:30 - 15:00 without a firewall those don't matter because you're going to be able to detect those anyway with a decent Discovery it's the ones that block all but a few ports that are hard to find and so out of those I looked at the most commonly responsive ports they don't have to be open they can be closed and send a reset and that works just as well and you see here many of the normal suspects ACP SMTP SSH some people look at this list and say hey where are the windows ports 135
• 15:00 - 15:30 139 those are really common but remember that I was only doing this based on heavily firewalled hosts and if you go through the trouble of setting up a firewall you better darn well block the windows ports so what I would advise is use some of these with sin probes and then some of the other ones uh with axe uh next we have UDP host Discovery and that one's simpler your strategy there is you want to find closed ports
• 15:30 - 16:00 because an open UDP Port normally won't even respond to the probe it'll just be like well I just got a blank packet don't know what to do with it just ignore it whereas closed ports will generally send a port unreachable packet which discloses that the host is live so I pick a high numbered close Port usually and then sometimes I'll do 53 as well because DNS is so popular with UDP that sometimes people allow it into their whole part of parts of their Network and so that can be effective as
• 16:00 - 16:30 well uh there's also icmp host Discovery methods offered by nmap and here the thing is some administrators like say google.com will say we're going to explicitly allow Echo requests because we don't consider ping packets a threat but only hackers use net mask request and timestamp request so we're going to block those however you also have administrators who kind of do the opposite they say oh I don't want those evil hackers to be able to ping me so they'll block the Ping requests but then
• 16:30 - 17:00 they'll forget that you can do the same thing with these other two so my suggestion is usually do an echo request plus one of these other two normally works pretty well uh we also have a new feature relatively called protocol ping which basically sends IP packets with various protocol headers and tries to expect a protocol unreachable message if the host is live and that can be useful I have haven't actually done the test to
• 17:00 - 17:30 see which Protocols are most commonly useful for this particular type of probe but by default we do icmp igmp and IP tunnel in IP so now I've talked about a lot of different Discovery techniques and which ones you might want to use but your question might be really how valuable is this it's going to take longer to scan if you add a bunch of Discovery techniques and so you have a leg legitimate question in how much of difference will it really make and again
• 17:30 - 18:00 instead of just guessing or making assumptions a good thing to do is test in this example I generate 50,000 IPS and then you can see I use the default ping scan and it chugs along and it finds 3,348 host up in about 1600 seconds which is about 27 minutes and so that's a lot of machines and you know it looks pretty successful but then I take that exact same list of 50,000 hsts and I add
• 18:00 - 18:30 a bunch more Discovery techniques we do the echo request the timestamp send probes to a bunch of ports act probes to a bunch of ports we set the source port to 53 in order to masquerade as DNS and it goes through and this time it finds 4,473 host up but it does take a bit longer so you have to ask yourself you know look at the data here it took almost three times as long but we found 34% more hosts and I think in most cases
• 18:30 - 19:00 if you want a comprehensive scan you're going to find that to be worthwhile so now I just have a plea basically about upgrading your end map and part of it is I'm sick of bug reports where it's like yes we fixed that in 2003 there are a lot of people who just don't seem to upgrade all that often and then they complain or they'll say hey the problem with nmap is it's obsolete there's it tells you what port numbers
• 19:00 - 19:30 are open but you don't know what services are behind them and nowadays everyone has they'll tunnel everything over HTTP in order to get through firewalls or whatnot and it's like hey we added version detection in 2003 you know just upgrade in addition I made a number of improvements uh to the performance uh of the system recently uh which you'll find valuable if you upgrade and then the question is what version should you upgrade to uh version 4.68 is the latest
• 19:30 - 20:00 release on our download page if you want to get even newer we have our subversion source code repository releases and you can find information at this URL or just go to the web page and you can track it down but for all the goods in this presentation you'll want to use the BHD co8 black hat Defcon release which you can find at this location and that contains the top ports feature and some of the other ones that I'm going to be talking about
• 20:00 - 20:30 So speaking of top ports uh this was another one of my big scans and here I wanted to determine the most commonly open TCP and UDP ports and again I got some data also contributed from organizations to look at representation of internal networks and then I took that data and I augmented the nmap services by which lists all the services
• 20:30 - 21:00 known by nmap and that enabled me to add a number of cool features first let's talk about the default scan ports in nmap 4.68 nmap would scan all the ports up to 10,24 plus it would scan all the ones it has a name for but the issue is that you know the I gave names to a bunch of ports you know many many many years ago many of which aren't even used uh really anymore and at the same time there are some ports that you see open more often
• 21:00 - 21:30 uh that turned out to not have names so with the new end map since it now has this frequency data it's able to just scan the top 1,000 ports for each protocol so you get better results in many cases since it has all these ports that the old one didn't have and it doesn't waste time scanning these ports that don't actually respond generally and at the same time it's a lot faster because it's only scanning a little more than half the ports so you'll find find you know a little increase in your scan
• 21:30 - 22:00 times from that but what it really makes a difference is the fast scan that's the traditional hyphen capital F option of end map which used to just say scan all the ports with a name but the major problem with that is hey by default we had 1,700 TCP ports with fast scan we had 1,200 you know that's not really fast that's kind of a small difference but nothing dramatic but that's all inmap could do because it didn't really know what port were common it only knew
• 22:00 - 22:30 which ones it had a name for with the new Services file nmap just scans the top 100 ports for each protocol and so you get usually an order of magnitude increase in speed which is helpful for TCP but it's even more helpful in many cases for UDP because I've seen a lot of people who basically don't even do their UDP scanning because they say oh it takes too long and it's hard to disambiguate the filtered versus the open ports and so they just pretend it doesn't exists but the attackers aren't
• 22:30 - 23:00 going to pretend it doesn't exist and so it's really important to figure out what's going on with this protocol and now let's look at an example of the difference that this makes here I'm doing a scan I say s u for UDP scan V to do the version detection and that's important for UDP scans because of that open versus filtered problem uh normally when nmap gets no response it doesn't know if the port's filtered or open and in the case of scan me.n map.org that's
• 23:00 - 23:30 the case with all of the ports so it's like great I got a report that said all the ports are either open or filtered that's no good and so with version detection and map has a database of probes it can send to each port and hopefully get a response which proves without a doubt that the port is open I say do a fast scan use aggressive timing against this machine that I maintain for people to scan and with 4.68 that took an hour and 2 minutes um it did find the
• 23:30 - 24:00 right data but that's still a long time to wait with the black hat Defcon release that same command took 6 minutes and 29 seconds because it was only scanning you know the most important ports and it knew what those were and it did find uh the open port then I optimized a bit more I said also add the version intensity zero flag which says only send these UDP probes for protocols that you know commonly listed on a certain Port so for 53 it'll only try
• 24:00 - 24:30 DNS for 161 only SNMP and with that that reduced the time to 13 seconds so the moral of the story is hey if you know what you're doing know what data you really need you can optimize your scan a bit and make it a lot faster in this case we got exactly the same data but instead of waiting an hour we waited 13 seconds uh which helps a lot uh tune features which are kind of derivative of those is the top ports feature which says hey you don't want to
• 24:30 - 25:00 just have to choose between the default of a thousand ports or a fast scan of 100 you can specify arbitrarily how many ports you want to scan and that leads you to the question of what will work best of to the top ports option and so I used empirical data again to say out of all these big scans how many of the open ports would I have found with different uh top ports values so if you just scan the top 10 ports which just goes really really lightning fast you get up almost
• 25:00 - 25:30 half the TCP ports with 100 which is the fast scan you get 73% of them whereas with a th000 which is the default you get 93% so that's pretty good to get 93% of the ports but you're only scanning less than 2% of the total 655,000 Port space so what I think a lot of pent testers will do is say hey I'm starting this engagement but I need some data to start with so they'll start a fast scan to scan the top 100 ports really quickly
• 25:30 - 26:00 and get that data and start working on it and while they're working on the initial data they'll have their super comprehensive all ports no ping scan going and then at the end of the scan of the big one they can just diff the results and see if there were any New Ports that the initial quick scan missed uh just in case you're interested uh these are the top 10 open TCP ports I found this differs from the previous chart because that was just responsive ports that could be open or closed uh 80
• 26:00 - 26:30 is the top no surprise as a security guy it's kind of depressing to see tnet open more often than SSH um a lot of that switches and routers and various devices um and here of course you do get the Microsoft ports because we're looking at open um similarly I have the data for UDP uh Microsoft kind of dominates this chart although you see some of the other normal suspects like I guess an MP and ntp and here's the UDP
• 26:30 - 27:00 Effectiveness uh of the different top ports values with UDP you get even a greater percentage open with a smaller uh value so here you get a 90 you know you get uh 90% with the top 100 ports whereas before we only got 73 with TCP uh here's another feature uh that we've added recently that I have to admit I have mixed Billings about um you know I'm kind of proud of nmap's
• 27:00 - 27:30 congestion control and other Technologies to try and figure out what scan speed will work best um but there are a lot of people who say hey I just want to specify a certain rate and have you scan at that speed and don't worry about if there are any packet drops or latency issues or whatnot just go at the speed I say so that I know exactly when it'll finish and it was basically for that one reason that a lot of people used uh scan Rand and unicorn scan and that type of scanner and so finally I
• 27:30 - 28:00 broke down and was like hey it's an easy an easy feature to add and even I found it uh to be pretty useful at times and in fact I used it during most of my internet scanning and then a feature that's even more new uh came about when the ISP call came saying I was melting their switches and so that's a maximum rate to say end map don't scan more than 300 packets per second or whatever you specify and so that uh made the ISP guys
• 28:00 - 28:30 a bit happy so here's an example of putting it all together um looking at kind of a typical type of the scans that I was doing and what options I was using I would say nmap I would give it the source IP address uh that I wanted to use for that particular scan I would specify debugging mode although really I found I used the runtime interaction feature more often when nmap is running some people don't know you can press D and the debugging level will increase and press it a few times and you'll really be scrolling the screen but
• 28:30 - 29:00 you'll see exactly what end map's doing right at that time then you can press capital d to turn it down say hey I'm done looking at this I don't want to fill up my log files uh turn it off for now I specified a low Mac scan delay because I didn't want to wait a long time for hosts that were rate limiting uh I did the log file feature with the new feature that says use Str strf time values so that it automatically puts the time and date in there I give it the
• 29:00 - 29:30 name of the file I want to read from I say don't do more than one retry uh for this case since I really want to do a big scan and make it go fast uh randomize the host in the scan group I do all the ports uh here's the host Discovery options I specified a reasonably Big Max host group because that's more efficient for large scans here I'm saying scan at at least 175 packets per second but don't scan at more than 300 so that's sort of an example of a
• 29:30 - 30:00 command that I sort of changed and changed and improved over time until I found one that worked pretty well so now with the time I have left I saved some to talk about some nmap news because some of these are features that are new and cool that may not reflect exactly relate to the large scale scanning but they're actually too cool to leave out so there are a few new features in map that I really wanted to talk about one is the nmap scripting engine which
• 30:00 - 30:30 is a thing that modularizes nmap and lets you say hey I want to write a little script that interrogates ports in a certain way uh in this case we do the HTML title for the websites it finds and there are now more than uh 50 scripts shipped with nmap everything from like who is Data to uh brute forcing pop three passwords you know there are all sorts of crazy things you can do with it I do have a quick demo of the M map scripting engine let
• 30:30 - 31:00 me see if I can find it it's a long command so I kind of cheated and uh put the actual command here but we're saying n map uh hyphen V in verbos mode don't ping uh do a UDP probe for Port 53 aggressive timing and we're going to do three scripts which uh I thought were kind of timely because they relate to uh Dan's DNS bug that
• 31:00 - 31:30 he'll be talking about on Sunday and so one of them just checks if a DNS server allows recursion the second one checks if it randomizes its source port numbers and the third one checks if it has a uh transaction ID that's randomized so those are the bugs that uh that people want to fix in order to reduce the cash poisoning issues and in this case I'm going to run it against uh one of black hats authoritative name servers and also
• 31:30 - 32:00 one of the authoritative name servers for sho.com the guys who put on the great shmon conference and so it does the port scan then it does the mmap scripting engine it takes it a little while but then it gives you your results right next to the port number it says that the black hat one basically refused recursion in both cases so it wasn't able to uh to interrogate them further the shukan it was recursive um but I'm happy to report that it was great in Source uh port
• 32:00 - 32:30 randomization and it was great in transaction ID randomization now I was going to show one of the many examples that fail miserably and maybe have a little challenge game to see who can poison the cash first but decided maybe that wouldn't be the most responsible thing um plus I've got a lot more good stuff to cover um one of the things I'm excited about is the new use nmap goey and a lot of people give me crap like
• 32:30 - 33:00 what I don't need no guey I've been using nmap 10 years and I know all of its 113 options by heart and I have to admit they have a point when you look at the old nmap Fe which frankly kind of sucked it basically just displayed your map output and instead of typing SS you uh pressed the button for sin scan whereas zenmap is a much more powerful interface and I'll give a quick demo of that basically just like nmap Fe it
• 33:00 - 33:30 could show you your normal output um it also has a tab that can say hey show me what each host has open or look at a service level and say show me the ones with HTTP or SSH and then it has a new experimental feature uh that we're adding and we only have in the subversion right now uh which basically says hey if you're going to call the dang tool n map Network mapper it ought to at least draw you a map
• 33:30 - 34:00 yeah thank you this is certainly the best feature for an eye candy perspective and it can be pretty neat it basically takes the scan that you did and it puts it in the center The Source host and then it in concentric circles around the center it shows each hop on the network and the machines that you scanned and you can take one and say Hey
• 34:00 - 34:30 you know show me what more data on this particular scan show me the ports that are open you can scan new machines and they'll get added to the graph and in terms of the biggest I candy aspect it's if you want to reenter the graph on a certain host that maybe makes it easier to read yes so maybe every once in a while that'll convince people to actual open
• 34:30 - 35:00 the uh the guey and try it out it also has a side benefit for me which is there are a lot of Windows users out there who have no idea how to do a command line scan or what command line even means I get so many mails saying hey I double clicked on nm. exex and it put this strange Black Box on the screen which then disappeared obviously nmap's totally broken um so maybe this will help them but on the
• 35:00 - 35:30 other hand maybe those people shouldn't be using nmap at all uh we have a second generation OS detection system uh which basically took all the things I learned with the first seven years of os detection and uh improved it and we're now up to, 1500 uh signatures with the new system so I'm hoping that'll help in terms of angularity you know it nmap users
• 35:30 - 36:00 basically can find every device you can possibly imagine so sure you've got your normal windows and Linux versions and the like but they find you know game consoles and pbxs and Network Power devices and all sorts of crazy things it's always fun to go through the submissions and see what people have uh version detection you know a lot of people like I said still don't know about it but hopefully the people who go to Defcon uh do a feature that is sometimes not known as
• 36:00 - 36:30 well as it could be is the reason feature basically if map tells you say a state is filtered you don't necessarily know is that because it sent me an icmp host unreachable packet is that because it got no response well the way to figure out what map's doing more is use hyphen hyen reason and then you can see hey this one's open because we got a synac this one's closed because we got a reset and that's a real good way to help understand and what end map's really doing and when that doesn't give you
• 36:30 - 37:00 enough information there's the packet Trace option so say in this previous scan I wasn't sure whether Port 25 mail and 113 is it the destination host that's sending those reset packets and I'm actually reaching it or is it a firewall sending some of them in between or is it a different case for each well by looking at the packet Trace option from a quick scan of just those two ports so you don't plug your screen too much you can look at things like the sequence number and windows number uh
• 37:00 - 37:30 what options they use U IP ID and that can often help figure out if it's the same host in both cases sending the packet uh which can be useful when you're trying to understand the firewalls and filtering systems in place um Advanced trace route you know it's trace route which isn't all that exciting but at least it does it better because nmap already knows what sorts of probes are likely to get through and it's also fast because then map can do it in parallel I made a number of
• 37:30 - 38:00 performance and accuracy improvements um there's a whole section on the man page uh showing all the different options you can use and what might help uh TCP and IP header options lets you specify things like Source routing um the uh record route option and some of you might be saying Source routing maybe that worked 15 years ago but come on there's no way that would ever work nowadays um but that's actually proved untrue in a number of cases I was
• 38:00 - 38:30 talking to a guy recently who was doing a test of a network and he was I guess in their conference room or the like and he was on a separate VLAN that could only contact one MCH uh a series of servers on their network but he couldn't contact all the client machines uh for the company just basically a little DMZ they enabled uh that they allowed access to from the conference room so he basically took one of those servers and said I want a loose Source route through that server to the destination machine
• 38:30 - 39:00 and he was able to get around that restriction that way which I thought was pretty cool another neat feature is called ncat and I shouldn't call it a feature because it's actually a whole new tool that I hope to ship with nmap and it's basically a modern interpretation of the netcat that we all know and love it basically supports virtually all of net cat's 1.10 features except the port scanner because I have another tool I like to use for that um but it also suggest supports a
• 39:00 - 39:30 lot of other uh cool new things like SSL both for communicating between netcat instances and to SSL htttp servers uh it supports IPv6 it works on Mac OS 10 on Windows on Linux on Unix it uh does connection brokering so if you set up a netcat listener in brokering mode then all of your machines behind Nats uh that want to connect to that netcat uh can do so through the broker
• 39:30 - 40:00 they can connect to that port and then talk to each other for command and control or whatever uh Port redirection is there are a lot of different tools for doing that now if you look around and get a specific tool but it's something I want to do often enough that I wanted to have it built in uh it can do proxying either as a client and do your stuff through a series of proxies or it can act as a proxy once you get onto the machine start it up as a proxy and then you can proxy through it to other machines uh shell execution uh
• 40:00 - 40:30 access control because you don't want anyone else connecting to your your net cap and it's something I've wanted for a long time and has been in development since 2005 uh right now it's a currently Dev lead is Chris Cater John one of the summer of code students and I think he may be here are you here Chris hey let's give a hand to Chris uh Chris has also added a lot of the
• 40:30 - 41:00 other uh features I demoed particularly the uh IP option uh ping discovery mode that was his idea and he put that in uh we have endi which is a simple tool but it does something that a lot of people have wanted for a long time which is taking two scans and diffing them so say I run a company Network I scan every day all the hosts in my cron tab I can then call endi and say May me the changes since
• 41:00 - 41:30 yesterday so any New Ports became open any new machines on the network any machines went down uh this will let you know and we have a python proof of concept right now in rbn and we're rewriting it uh in C since it proved to be useful and people want it uh the C version will work even better another thing is my map book which I've been working on for years uh so long that people have been comparing it to Duke Nukem Forever and the like in uh
• 41:30 - 42:00 greatest vaporware so I was like dang it I'm not gonna go to Defcon and black hat empty-handed again especially after last year telling people oh it's almost done uh so I work pretty hard uh to get it ready and I'm happy to say that I uh finally have it now and did a pre-release here thank thanks I uh I hope it does a good job at not just telling you what
• 42:00 - 42:30 options there are for end map but also uh how to use them effectively to scan your networks and so I uh last minute printed 170 copies and brought them here so I could tell people about them in my talk and they could go pick them up but I'm afraid they were sold like after an hour this morning and so they're all gone now but um I hope as soon as I can I'm going to get it on Amazon and the like and also so half of it is already available online for free at nm.org book
• 42:30 - 43:00 and that's also where I'll be uh putting the details of the launch of the book and um you can also join the nmap hackers list if you're not a member it's a pretty low volume list I think I've sent three messages this year as opposed to mmap Dev which gets thousands of them but mmap hackers you can join and I'll send you the latest news uh when it's ready I also wanted to do a slide because sometimes I get you know way more credit than I deserve uh for map
• 43:00 - 43:30 just because I created it way back uh but it's actually a project uh that's very fortunate to have tons and tons of contributors and I couldn't do it without them this is an example of just the people who've contributed significantly since version 4.50 which was nine months ago so you can see that uh the map project is really lucky to have a lot of volunteers who uh help out uh greatly now with that uh out of the way I think
• 43:30 - 44:00 I have time for maybe two or three questions before we'll go to the question room if anyone has more so who's got a question for me how do you feel yeah I don't feel very good about Germany and the UK and other countries that have put laws which tend to people suggest May ban tools like nmap that can be used for good things even though um attackers might use them as well and I think that's really dangerous I mean the typical analogy is Banning a hammer you
• 44:00 - 44:30 know by blocking it you ensure that the good guys aren't able to use it to improve their networks and personally since I like to give talks in places like Germany and England um that that can be a potential issue because I'd hate to get you know busted in Germany and these laws say things like what's it designed for what's the motivation and so how do I convince a judge that my motivation was good when you know I can't even read German to read the law
• 44:30 - 45:00 so those are definitely a scary issue and thing that that I'm glad that uh groups are trying to fight even though we've had some losses there do I have another question I can hardly see because of a giant bright light shining at me reason option oh good point uh the reason an option we sort of always have that field there so
• 45:00 - 45:30 it won't have any uh performance impact at all to add that so in many cases just like hyphen V and like hyphen T4 it becomes one of the things I almost always use all right so if anyone has any other questions I'm going to be in room 103 which is just across the hall and I'd be happy to answer them there thank you very much