Sublert – Security And Reconnaissance Tool Which Leverages Certificate Transparency To Automatically Monitor New Subdomains Deployed By Specific Organizations And Issued TLS/SSL Certificate

Sublert is a security and reconnaissance tool that was written in Python to leverage certificate transparency for the sole purpose of monitoring new subdomains deployed by specific organizations and issued TLS/SSL certificate. The tool is supposed to be scheduled to run periodically at fixed times, dates, or intervals (Ideally each day). New identified subdomains will be sent to Slack workspace with a notification push. Furthermore, the tool performs DNS resolution to determine working subdomains.RequirementsVirtual Private Server (VPS) running on Unix. (I personally use digitalOcean)Python 2.x or 3.x.Free Slack workspace.Installation & ConfigurationPlease refer to below article for a detailed technical explanation:https://medium.com/@yassineaboukir/automated-monitoring-of-subdomains-for-fun-and-profit-release-of-sublert-634cfc5d7708Usage Short Form Long Form Description -u –url Adds a domain to monitor. E.g: yahoo.com. -d –delete Domain to remove from the monitored list. E.g: yahoo.com. -a –list Listing all monitored domains. -t –threads Number of concurrent threads to use (Default: 20). -r –resolve Perform DNS resolution. -l –logging Enable Slack-based error logging. -m –reset Reset everything. Feedback and issues?If you have any feedback, anything that you want to see implemented or running into issues using Sublert, please feel free to file an issue on https://github.com/yassineaboukir/sublert/issuesDownload Sublert

Link: http://feedproxy.google.com/~r/PentestTools/~3/AGTDH5ASc-U/sublert-security-and-reconnaissance.html

IPRotate – Extension For Burp Suite Which Uses AWS API Gateway To Rotate Your IP On Every Request

Extension for Burp Suite which uses AWS API Gateway to change your IP on every request.More info: https://rhinosecuritylabs.com/aws/bypassing-ip-based-blocking-aws/DescriptionThis extension allows you to easily spin up API Gateways across multiple regions. All the Burp Suite traffic for the targeted host is then routed through the API Gateway endpoints which causes the IP to be different on each request. (There is a chance for recycling of IPs but this is pretty low and the more regions you use the less of a chance.)This is useful to bypass different kinds of IP blocking like bruteforce protection that blocks based on IP, API rate limiting based on IP or WAF blocking based on IP etc.UsageSetup Jython in Burp SuiteInstall the boto3 module for Python 2 pip install boto3Ensure you have a set of AWS keys that have full access to the API Gateway service. This is available through the free tier of AWS.Insert the credentials into the fields.Insert the target domain you wish to target.Select HTTPS if the domain is hosted over HTTPS.Select all the regions you want to use.(The more you use the larger the IP pool will be)Click “Enable".Once you are done ensure you click disable to delete all the resources which were started.If you want to check on the resources and enpoints that were started or any potential errors you can look at the output console in Burp.The Burp UIExample of how the requests lookSetupMake sure you have Jython installed and add IPRotate.py through the Burp Extension options.Previous ResearchAfter releasing this extension it was pointed out that there has been other research in this area using AWS API Gateway to hide an IP address. There is some awesome research and tools by @ustayready @ryHanson and @rmikehodges using this technique.Be sure to check them out too:https://github.com/ustayready/fireproxhttps://github.com/rmikehodges/hideNsneakDownload IPRotate_Burp_Extension

Link: http://feedproxy.google.com/~r/PentestTools/~3/t5h8C83KVMM/iprotate-extension-for-burp-suite-which.html

LDAPDomainDump – Active Directory Information Dumper Via LDAP

Active Directory information dumper via LDAPIntroductionIn an Active Directory domain, a lot of interesting information can be retrieved via LDAP by any authenticated user (or machine). This makes LDAP an interesting protocol for gathering information in the recon phase of a pentest of an internal network. A problem is that data from LDAP often is not available in an easy to read format.ldapdomaindump is a tool which aims to solve this problem, by collecting and parsing information available via LDAP and outputting it in a human readable HTML format, as well as machine readable json and csv/tsv/greppable files.The tool was designed with the following goals in mind:Easy overview of all users/groups/computers/policies in the domainAuthentication both via username and password, as with NTLM hashes (requires ldap3 >=1.3.1)Possibility to run the tool with an existing authenticated connection to an LDAP service, allowing for integration with relaying tools such as impackets ntlmrelayxThe tool outputs several files containing an overview of objects in the domain:domain_groups: List of groups in the domaindomain_users: List of users in the domaindomain_computers: List of computer accounts in the domaindomain_policy: Domain policy such as password requirements and lockout policydomain_trusts: Incoming and outgoing domain trusts, and their propertiesAs well as two grouped files:domain_users_by_group: Domain users per group they are member ofdomain_computers_by_os: Domain computers sorted by Operating SystemDependencies and installationRequires ldap3 > 2.0 and dnspythonBoth can be installed with pip install ldap3 dnspythonThe ldapdomaindump package can be installed with python setup.py install from the git source, or for the latest release with pip install ldapdomaindump.UsageThere are 3 ways to use the tool:With just the source, run python ldapdomaindump.pyAfter installing, by running python -m ldapdomaindumpAfter installing, by running ldapdomaindumpHelp can be obtained with the -h switch:usage: ldapdomaindump.py [-h] [-u USERNAME] [-p PASSWORD] [-at {NTLM,SIMPLE}] [-o DIRECTORY] [–no-html] [–no-json] [–no-grep] [–grouped-json] [-d DELIMITER] [-r] [-n DNS_SERVER] [-m] HOSTNAMEDomain information dumper via LDAP. Dumps users/computers/groups andOS/membership information to HTML/JSON/greppable output.Required options: HOSTNAME Hostname/ip or ldap://host:port connection string to connect to (use ldaps:// to use SSL)Main options: -h, –help show this help message and exit -u USERNAME, –user USERNAME DOMAIN\username for authentication, leave empty for anonymous authentication -p PASSWORD, –password PASSWORD Password or LM:NTLM hash, will prompt if not specified -at {NTLM,SIMPLE}, –authtype {NTLM,SIMPLE} Authentication type (NTLM or SIMPLE, default: NTLM)Output options: -o DIRECTORY, –outdir DIRECTORY Directory in which the dump will be saved (default: current) –no-html Disable HTML output –no-json Disable JSON output –no-grep Disable Greppable output –grouped-json Also write json files for grouped files (default: disabled) -d DELIMITER, –delimiter DELIMITER Field delimiter for greppable output (default: tab)Misc options: -r, –resolve Resolve computer hostnames (might take a while and cause high traffic on large networks) -n DNS_SERVER, –dns-server DNS_SERVER Use custom DNS resolver instead of system DNS (t ry a domain controller IP) -m, –minimal Only query minimal set of attributes to limit memmory usageOptionsAuthenticationMost AD servers support NTLM authentication. In the rare case that it does not, use –authtype SIMPLE.Output formatsBy default the tool outputs all files in HTML, JSON and tab delimited output (greppable). There are also two grouped files (users_by_group and computers_by_os) for convenience. These do not have a greppable output. JSON output for grouped files is disabled by default since it creates very large files without any data that isn’t present in the other files already.DNS resolvingAn important option is the -r option, which decides if a computers DNSHostName attribute should be resolved to an IPv4 address. While this can be very useful, the DNSHostName attribute is not automatically updated. When the AD Domain uses subdomains for computer hostnames, the DNSHostName will often be incorrect and will not resolve. Also keep in mind that resolving every hostname in the domain might cause a high load on the domain controller.Minimizing network and memory usageBy default ldapdomaindump will try to dump every single attribute it can read to disk in the .json files. In large networks, this uses a lot of memory (since group relationships are currently calculated in memory before being written to disk). To dump only the minimal required attributes (the ones shown by default in the .html and .grep files), use the –minimal switch.Visualizing groups with BloodHoundLDAPDomainDump includes a utility that can be used to convert ldapdomaindumps .json files to CSV files suitable for BloodHound. The utility is called ldd2bloodhound and is added to your path upon installation. Alternatively you can run it with python -m ldapdomaindump.convert or with python ldapdomaindump/convert.py if you are running it from the source. The conversion tool will take the users/groups/computers/trusts .json file and convert those to group_membership.csv and trust.csv which you can add to BloodHound.Download Ldapdomaindump

Link: http://feedproxy.google.com/~r/PentestTools/~3/NVmrl3qPNRU/ldapdomaindump-active-directory.html

PoshC2 – C2 Server and Implants

PoshC2 is a proxy aware C2 framework that utilises Powershell and/or equivalent (System.Management.Automation.dll) to aid penetration testers with red teaming, post-exploitation and lateral movement. Powershell was chosen as the base implant language as it provides all of the functionality and rich features without needing to introduce multiple third party libraries to the framework.In addition to the Powershell implant, PoshC2 also has a basic dropper written purely in Python that can be used for command and control over Unix based systems such as Mac OS or Ubuntu.The server-side component is written in Python for cross-platform portability and speed, a Powershell server component still exists and can be installed using the ‘Windows Install’ as shown below but will not be maintained with future updates and releases.Linux Install Python3Automatic install for Python3 using curl & bashcurl -sSL https://raw.githubusercontent.com/nettitude/PoshC2_Python/master/Install.sh | bashManual install Python3wget https://raw.githubusercontent.com/nettitude/PoshC2_Python/master/Install.shchmod +x ./Install.sh./Install.shLinux Install Python2 – stable but unmaintainedAutomatic install for Python2 using curl & bashcurl -sSL https://raw.githubusercontent.com/nettitude/PoshC2_Python/python2/Install.sh | bashManual install Python2wget https://raw.githubusercontent.com/nettitude/PoshC2_Python/python2/Install.shchmod +x ./Install.sh./Install.shWindows InstallInstall Git and Python (and ensure Python is in the PATH), then run:powershell -exec bypass -c “IEX (New-Object System.Net.WebClient).DownloadString(‘https://raw.githubusercontent.com/nettitude/PoshC2_Python/master/Install.ps1’)"Using older versionsYou can use an older version of PoshC2 by referencing the appropriate tag. You can list the tags for the repository by issuing:git tag –listor viewing them online.Then you can use the install one-liner but replace the branch name with the tag:curl -sSL https://raw.githubusercontent.com/nettitude/PoshC2_Python//Install.sh | bashFor example:curl -sSL https://raw.githubusercontent.com/nettitude/PoshC2_Python/v4.8/Install.sh | bashOfflineIf you have a local clone of PoshC2 you can change the version that is in use by just checking out the version you want to use:git reset –hard <tag name>For example:git reset –hard v4.8However note that this will overwrite any local changes to files, such as Config.py and you may have to re-run the install script for that version or re-setup the environment appropriately.Running PoshC2Edit the config file by running posh-config to open it in $EDITOR. If this variable is not set then it defaults to vim, or you can use –nano to open it in nano.Run the server using posh-server or python3 -u C2Server.py | tee -a /var/log/poshc2_server.logOthers can view the log using posh-log or tail -n 5000 -f /var/log/poshc2_server.logInteract with the implants using the handler, run by using posh or python3 ImplantHandler.pyInstalling as a serviceInstalling as a service provides multiple benefits such as being able to log to service logs, viewing with journalctl and automatically starting on reboot.Add the file in systemd (this is automatically done via the install script)cp poshc2.service /lib/systemd/system/poshc2.serviceStart the serviceposh-serviceView the log:posh-logOr alternatively us journalctl (but note this can be rate limited)journalctl -n 20000 -u poshc2.service -f –output catNote that re-running posh-service will restart the posh-service. Running posh-service will automatically start to display the log, but Ctrl-C will not stop the service only quit the log in this case posh-log can be used to re-view the log at any point. posh-stop-service can be used to stop the service.Issues / FAQsIf you are experiencing any issues during the installation or use of PoshC2 please check the known issues below and the open issues tracking page within GitHub. If this page doesn’t have what you’re looking for please open a new issue and we will try to resolve the issue asap.If you are looking for tips and tricks on PoshC2 usage and optimisation, you are welcome to join the slack channel below.License / Terms of UseThis software should only be used for authorised testing activity and not for malicious use.By downloading this software you are accepting the terms of use and the licensing agreement.DocumentationWe maintain PoshC2 documentation over at https://poshc2.readthedocs.io/en/latest/Find us on #Slack – poshc2.slack.com (to request an invite send an email to labs@nettitude.com)Known issuesError encrypting value: object typeIf you get this error after installing PoshC2 it is due to dependency clashes in the pip packages on the system.Try creating a virtualenv in python and re-install the requirements so that the exact versions specified are in use for PoshC2. Make sure you deactivate when you’ve finished in this virtualenv.For example:pip install virtualenvvirtualenv /opt/PoshC2_Python/source /opt/PoshC2_Python/bin/activatepip install -r requirements.txtpython C2Server.pyNote anytime you run PoshC2 you have to reactivate the virtual environment and run it in that.The use of a virtual environment is abstracted if you use the posh- scripts on *nix.Download PoshC2_Python

Link: http://feedproxy.google.com/~r/PentestTools/~3/cYFi81W7lAw/poshc2-c2-server-and-implants.html

Hacktronian – All In One Hacking Tool For Linux & Android

***Pentesing Tools That All Hacker Needs.***HACKTRONIAN Menu :Information GatheringPassword AttacksWireless TestingExploitation ToolsSniffing & SpoofingWeb HackingPrivate Web HackingPost ExploitationInstall The HACKTRONIANInformation Gathering:NmapSetoolkitPort ScanningHost To IPwordpress userCMS scannerXSStrikeDork – Google Dorks Passive Vulnerability AuditorScan A server’s UsersCripsPassword Attacks:CuppNcrackWireless Testing:reaverpixiewpsFluxionExploitation Tools:ATSCANsqlmapShellnoobcommixFTP Auto Bypassjboss-autopwnSniffing & Spoofing:SetoolkitSSLtrippyPISHERSMTP MailerWeb Hacking:Drupal HackingInurlbrWordpress & Joomla ScannerGravity Form ScannerFile Upload CheckerWordpress Exploit ScannerWordpress Plugins ScannerShell and Directory FinderJoomla! 1.5 – 3.4.5 remote code executionVbulletin 5.X remote code executionBruteX – Automatically brute force all services running on a targetArachni – Web Application Security Scanner FrameworkPrivate Web Hacking:Get all websitesGet joomla websitesGet wordpress websitesControl Panel FinderZip Files FinderUpload File FinderGet server usersSQli ScannerPorts Scan (range of ports)ports Scan (common ports)Get server InfoBypass CloudflarePost Exploitation:Shell CheckerPOETWeemanInstallation in Linux:This Tool Must Run As ROOT !!!git clone https://github.com/thehackingsage/hacktronian.gitcd hacktronianchmod +x install.sh./install.shThat’s it.. you can execute tool by typing hacktronianInstallation in Android:Open Termuxpkg install gitpkg install pythongit clone https://github.com/thehackingsage/hacktronian.gitcd hacktronianchmod +x hacktronian.pypython2 hacktronian.pyVideo Tutorial :Download Hacktronian

Link: http://feedproxy.google.com/~r/PentestTools/~3/yV_fdYg2NkU/hacktronian-all-in-one-hacking-tool-for.html

Pyshark – Python Wrapper For Tshark, Allowing Python Packet Parsing Using Wireshark Dissectors

Python wrapper for tshark, allowing python packet parsing using wireshark dissectors.Extended documentation: http://kiminewt.github.io/pysharkPython2 deprecation – This package no longer supports Python2. If you wish to still use it in Python2, you can:Use version 0.3.8Install pyshark-legacy via pypiClone the pyshark-legacy [repo (https://github.com/KimiNewt/pyshark-legacy)], where bugfixes will be applied.Looking for contributors – for various reasons I have a hard time finding time to maintain and enhance the package at the moment. Any pull-requests will be reviewed and if any one is interested and is suitable, I will be happy to include them in the project. Feel free to mail me at dorgreen1 at gmail.There are quite a few python packet parsing modules, this one is different because it doesn’t actually parse any packets, it simply uses tshark’s (wireshark command-line utility) ability to export XMLs to use its parsing.This package allows parsing from a capture file or a live capture, using all wireshark dissectors you have installed. Tested on windows/linux.InstallationAll PlatformsSimply run the following to install the latest from pypipip install pysharkOr install from the git repository:git clone https://github.com/KimiNewt/pyshark.gitcd pyshark/srcpython setup.py installMac OS XYou may have to install libxml which can be unexpected. If you receive an error from clang or an error message about libxml, run the following:xcode-select –installpip install libxmlYou will probably have to accept a EULA for XCode so be ready to click an “Accept" dialog in the GUI.UsageReading from a capture file:>>> import pyshark>>> cap = pyshark.FileCapture(‘/tmp/mycapture.cap’)>>> cap>>> print cap[0]Packet (Length: 698)Layer ETH: Destination: BLANKED Source: BLANKED Type: IP (0x0800)Layer IP: Version: 4 Header Length: 20 bytes Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport)) Total Length: 684 Identification: 0x254f (9551) Flags: 0x00 Fragment offset: 0 Time to live: 1 Protocol: UDP (17) Header checksum: 0xe148 [correct] Source: BLANKED Destination: BLANKED …Other optionsparam keep_packets: Whether to keep packets after reading them via next(). Used to conserve memory when reading large caps.param input_file: Either a path or a file-like object containing either a packet capture file (PCAP, PCAP-NG..) or a TShark xml.param display_filter: A display (wireshark) filter to apply on the cap before reading it.param only_summaries: Only produce packet summaries, much faster but includes very little informationparam disable_protocol: Disable detection of a protocol (tshark > version 2)param decryption_key: Key used to encrypt and decrypt captured traffic.param encryption_type: Standard of encryption used in captured traffic (must be either ‘WEP’, ‘WPA-PWD’, or ‘WPA-PWK’. Defaults to WPA-PWK.param tshark_path: Path of the tshark binaryReading from a live interface:>>> capture = pyshark.LiveCapture(interface=’eth0′)>>> capture.sniff(timeout=50)>>> capture<LiveCapture (5 packets)>>>> capture[3]<UDP/HTTP Packet>for packet in capture.sniff_continuously(packet_count=5): print ‘Just arrived:’, packetOther optionsparam interface: Name of the interface to sniff on. If not given, takes the first available.param bpf_filter: BPF filter to use on packets.param display_filter: Display (wireshark) filter to use.param only_summaries: Only produce packet summaries, much faster but includes very little informationparam disable_protocol: Disable detection of a protocol (tshark > version 2)param decryption_key: Key used to encrypt and decrypt captured traffic.param encryption_type: Standard of encryption used in captured traffic (must be either ‘WEP’, ‘WPA-PWD’, or ‘WPA-PWK’. Defaults to WPA-PWK).param tshark_path: Path of the tshark binaryparam output_file: Additionally save captured packets to this file.Reading from a live interface using a ring buffer>>> capture = pyshark.LiveRingCapture(interface=’eth0′)>>> capture.sniff(timeout=50)>>> capture<LiveCapture (5 packets)>>>> capture[3]<UDP/HTTP Packet>for packet in capture.sniff_continuously(packet_count=5): print ‘Just arrived:’, packetOther optionsparam ring_file_size: Size of the ring file in kB, default is 1024param num_ring_files: Number of ring files to keep, default is 1param ring_file_name: Name of the ring file, default is /tmp/pyshark.pcapparam interface: Name of the interface to sniff on. If not given, takes the first available.param bpf_filter: BPF filter to use on packets.param display_filter: Display (wireshark) filter to use.param only_summaries: Only produce packet summaries, much faster but includes very little informationparam disable_protocol: Disable detection of a protocol (tshark > version 2)param decryption_key: Key used to encrypt and decrypt captured traffic.param encryption_type: Standard of encryption used in captured traffic (must be either ‘WEP’, ‘WPA-PWD’, or ‘WPA-PWK’. Defaults to WPA-PWK).param tshark_path: Path of the tshark binaryparam output_file: Additionally save captured packets to this file.Reading from a live remote interface:>>> capture = pyshark.RemoteCapture(‘192.168.1.101’, ‘eth0’)>>> capture.sniff(timeout=50)>>> captureOther optionsparam remote_host: The remote host to capture on (IP or hostname). Should be running rpcapd.param remote_interface: The remote interface on the remote machine to capture on. Note that on windows it is not the device display name but the true interface name (i.e. \Device\NPF_..).param remote_port: The remote port the rpcapd service is listening onparam bpf_filter: A BPF (tcpdump) filter to apply on the cap before reading.param only_summaries: Only produce packet summaries, much faster but includes very little informationparam disable_protocol: Disable detection of a protocol (tshark > version 2)param decryption_key: Key used to encrypt and decrypt captured traffic.param encryption_type: Standard of encryption used in captured traffic (must be either ‘WEP’, ‘WPA-PWD’, or ‘WPA-PWK’. Defaults to WPA-PWK).param tshark_path: Path of the tshark binaryAccessing packet data:Data can be accessed in multiple ways. Packets are divided into layers, first you have to reach the appropriate layer and then you can select your field.All of the following work:>>> packet[‘ip’].dst192.168.0.1>>> packet.ip.src192.168.0.100>>> packet[2].src192.168.0.100To test whether a layer is in a packet, you can use its name:>>> ‘IP’ in packetTrueTo see all possible field names, use the packet.layer.field_names attribute (i.e. packet.ip.field_names) or the autocomplete function on your interpreter.You can also get the original binary data of a field, or a pretty description of it:>>> p.ip.addr.shownameSource or Destination Address: 10.0.0.10 (10.0.0.10)# And some new attributes as well:>>> p.ip.addr.int_value167772170>>> p.ip.addr.binary_value’\n\x00\x00\n’Decrypting packet capturesPyshark supports automatic decryption of traces using the WEP, WPA-PWD, and WPA-PSK standards (WPA-PWD is the default).>>> cap1 = pyshark.FileCapture(‘/tmp/capture1.cap’, decryption_key=’password’)>>> cap2 = pyshark.LiveCapture(interface=’wi0′, decryption_key=’password’, encryption_type=’wpa-psk’)A tuple of supported encryption standards, SUPPORTED_ENCRYPTION_STANDARDS, exists in each capture class.>>> pyshark.FileCapture.SUPPORTED_ENCRYPTION_STANDARDS(‘wep’, ‘wpa-pwd’, ‘wpa-psk’)>>> pyshark.LiveCapture.SUPPORTED_ENCRYPTION_STANDARDS(‘wep’, ‘wpa-pwd’, ‘wpa-psk’)Download Pyshark

Link: http://feedproxy.google.com/~r/PentestTools/~3/eTNAbeSDlQw/pyshark-python-wrapper-for-tshark.html

Applepie – A Hypervisor For Fuzzing Built With WHVP And Bochs

Hello! Welcome to applepie! This is a tool designed for fuzzing, introspection, and finding bugs! This is a hypervisor using the Windows Hypervisor Platform API present in recent versions of Windows (specifically this was developed and tested on Windows 10 17763). Bochs is used for providing deep introspection and device emulation.The Windows Hypervisor Platform API (WHVP) is an API set for accessing Hyper-V’s hypervisor abilities. This API makes it easy for us to implement a virtual machine all in user-space without any special drivers or permissions needed.Recent Feature DemoBinary Coverage ExampleWhat is this for?This is a tool designed for fuzzing and introspection during security research. By using a hypervisor common fuzzing techniques can be applied to any target, kernel or userland. This environment allows fuzzing of whole systems without a need for source of the target. At the hypervisor level code coverage can be gathered, and if needed Bochs emulation can be used to provide arbitrary introspection in an emulation environment. This coverage information can be used to figure out the effectiveness of the fuzz cases. A fuzz case that caused an increase in coverage can be saved as it was an interesting case. This input can be used later, built on by new corruptions.Snapshot fuzzing is the primary use of this tool. Where you take a snapshot of a system in a certain state, and save it off. This snapshot can then be loaded up for fuzzing, where a fuzz case is injected, and it’s resumed. Since the VM can be reset very cheaply, the VM can be reset often. If it takes Word 5 seconds to boot, but you can snapshot it right as it reads your file, you can cut the fuzz case down to only what is relevant to an input. This allows for a very tight loop of fuzzing without needing to have access to source. Since the VM’s are entirely separate systems, many can be run in parallel to allow scaling to all cores.Currently this tool only supports gathering code coverage, dynamic symbol downloading for Windows, and symbol/module parsing for Windows targets as well. Adding fuzzing support will be quite soon.Development cycleGiven I’ve written almost all the features here before (coverage, fuzzing, fast resets, etc). I expect this project should pretty quickly become ready for fuzzing, unless I get distracted :DI’m aiming for end-of-January for coverage (done!), feedback, module listings (done!), process lists, fast resets, and symbol support (done!). Which would make it a very capable fuzzer.OS SupportThe main supported target is modern Windows 10. Windows targets have downloading of symbols from the symbol store. This allows for symbolic coverage in Windows targets out of the box. However, the code is written in a way that Linux enlightenment can easily be added.Without any enlightment, any OS that boots can still be fuzzed and basic coverage can be gathered.Before reporting OS support issues please validate that the issue is in the hypervisor/changes to Bochs by trying to boot your target using standard prebuilt Bochs with no hypervisor. Bochs is not commonly used and can frequently have breaking bugs for even common things like booting Linux. Especially with the rapid internal changes to CPUID/MSR usages with Spectre/Meltdown mitigations going into OSes.IssuesSee the issues page on Github for a list of issues. I’ve seeded it with a few already. Some of these need to be addressed quickly before fuzzing development starts.BuildingBuild PrereqsTo build this you need a few things:Recently updated MSVC compiler (Visual Studio 2017)Nightly Rust (https://rustup.rs/ , must be nightly)Python (I used 3 but 2 should work too)64-bit cygwin with autoconf and GNU make packages installedHyper-V installed and a recent build of Windows 10MSVCInstall Visual Studio 2017 and make sure it’s updated. We’re using some bleeding edge APIs, headers, and libraries here.I was using cl.exe version: Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27025.1 for x64 And SDK version 10.0.17763.0Nightly RustInstall Rust via https://rustup.rs/. I used rustc 1.32.0-nightly (b3af09205 2018-12-04)Make sure you install the x86_64-pc-windows-msvc toolchain as only 64-bit is supported for this project.Make sure cargo is in your path. This should be the default.PythonGo grab python https://www.python.org/ and make sure it’s in your PATH such that python can be invoked.CygwinInstall 64-bit Cygwin (https://www.cygwin.com/setup-x86_64.exe) specifically to C:\cygwin64. When installing Cygwin make sure you install the autoconf and make packages.Hyper-VGo into “Turn Windows features on or off" and tick the checkbox next to "Hyper-V" and "Windows Hypervisor Platform". This requires of course that your computer supports Hyper-V.Step-by-step build processThis install process guide was verified on the following:Clean install of Windows 10, Build 17763rustc 1.33.0-nightly (8e2063d02 2019-01-07)Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27025.1 for x64Visual Studio Community 2017 version 15.9.4applepie commit `f84c084feb487e2e7f31f9052a4ab0addd2c4cf9`Python 3.7.2 x64git version 2.20.1.windows.1Make sure Windows 10 is fully up to date We use some bleeding edge features with WHVP and only latest Windows 10 is testedIn "Turn Windows features on or off" Tick "Hyper-V"Tick "Windows Hypervisor Platform"Click ok to install and rebootInstall VS Community 2017 and updated Desktop development with C++Install Rust nightly for x86_64-pc-windows-msvcInstall Git Configure git to checkout as-is, commit unix-styleIf git converts on checkout the ./configure script will fail for Bochs due to CRLF line endingsThis is core.autocrlf=inputYou can also use checkout as-is, commit as-isThis is core.autocrlf=falseInstall Cygwin x64 via setup-x86_64.exe Install to "C:\cygwin64"Install autoconf package (autoconf package)Install GNU make (make package)Install Python I installed Python 3 x64 and added to PATHPython 2 and 32-bit versions should be fine, we just use Python for our build scriptOpen a "x64 Native Tools Command Prompt for VS 2017"Checkout applepie via git clone https://github.com/gamozolabs/applepiecd into applepieRun python build.py This will first check for some basic system requirementsIt will build the Rust bochservisor DLLIt will then configure Bochs via autoconfIt will then build Bochs with GNU make from CygwinThis initial build process may take about 2 minutes, on a modern machine it’s likely 20-30 seconds.Actually BuildingJust run python build.py from the root directory of this project. It should check for sanity of the environment and everything should "just work".CleaningRun python build.py clean to clean Bochs and Rust binaries.Run python build.py deepclean to completely remove all Bochs and Rust binaries, it also removes all the configuration for Bochs. Use this if you reconfigure Bochs in some way.UsageRead up on Bochs configuration to figure out how to set up your environment. We have a few requirements, like sync=none, ips=1000000, and currently single processor support only. These are enforced inside of the code itself to make sure you don’t shoot yourself in the foot.Use the included bochservisor_test\bochsrc.bxrc and bochservisor_test_real\bochsrc.bxrc configurations as examples. bochservisor_test_real is likely the most up to date config you should look at as reference.CoverageWindows targets have module list enlightenment, which allows us to see the listings for all the modules in the context we are running in. With this we can convert the instruction addresses to module + offset. This module + offset helps keep coverage information between fuzz cases where ASLR state changes. It also allows for the module to be colored in a tool like IDA to visually see what code has been hit.For Windows targets, symbols will be dynamically downloaded from the symbol store using your _NT_SYMBOL_PATH and using symchk. Without symchk in the path it will silently fail. With symbols a nice human-readable version of coverage can be saved for viewing. Further, with private symbols the coverage can be converted to source:line such that source code can be colored.TestsOkay there aren’t really tests, but there’s bochservisor_test which is a tiny OS that just verifies that everything boots with the hypervisor.There’s then bochservisor_test_real which is a configuration I use for things like Windows/Linux. This is the one that will probably get updated most frequently.ArchitectureBasicsThis codebase introduces a small amount of code to Bochs to allow modular access to CPU context, guest physical to their backing memory, and stepping both device and CPU state.The main code you want to look at is in lib.rs in the bochservisor Rust project.CPU LoopIn the main CPU loop of Bochs we instead LoadLibrary() to load the bochservisor DLL. This DLL exports one routine which is the Rust CPU loop which will be invoked.Bochs will pass a structure to this bochs_cpu_loop routine which will contain function pointers to get information from Bochs and to step the device and CPU state in it.MMIO / I/OWhen MMIO or I/O occurs, the hypervisor will exit with a memory fault or an I/O instruction fault. While WHVP does provide an emulation API it’s really lacking and not sufficient.Rather we use Bochs which is already there and step through a few instructions. By keeping the hypervisor CPU state in sync with Bochs we can dynamically switch between hypervisor and emulation at any time (or at least we should be able to).This means that the full hypervisor state is always in sync with Bochs and thus things like Bochs snapshots should work as normal and could be booted without the hypervisor (except maybe some CPUID state which needs to be stored in the snapshot info).When MMIO or I/O occurs we run a certain number of instructions under emulation rather than just emulating one. Due to the API costs of entering and exiting the hypervisor, and the likelihood that similar MMIO operations occur next to others, we step a few instructions. This allows use to reduce the overhead of the API and reduces the VMEXIT frequency. This is a tunable number but what is in the codebase is likely there for a reason.InterruptsInterrupts we handle in a really interesting way. Rather than scheduling interrupts to be delivered to the hypervisor we handle all interrupts in Bochs emulation itself. Things like exceptions that happen inside of the hypervisor entirely of course are not handled by Bochs.This also gives us features that WHVP doesn’t support, like SMIs (for SMM). Bochs’s BIOS uses SMM by default and without SMI support a custom BIOS needs to be built. I did this in my first iteration of this… do not recommend.FutureThis project is designed for fuzzing, however it’s so new (only a few days old) that it has none of these features.Some of the first things to come will be:Evaluate threadingWe could potentially have Bochs device stuff running in one thread in a loop in real-time, and another thread running the hypervisor. Async events would be communicated via IPC and would allow for the devices to be updated while execution is in the guest.Currently everything happens in one thread which means the hypervisor must exit on an interval to make sure we can step devices. It’s as if we wrote our own scheduler.This might be a bit faster, but it also increases complexity and adds the potential for race issues. It’s hard to say if this will ever happen.Code coverageI’m not sure which method I’ll use to gather code coverage, but there will be at least a few options. Spanning from accurate, to fast, etc. All these coverage mechanisms will be system level and will not require source or symbols of targets.Guest enlightenmentParsing of OS structures to get primitive information such as process listings, module lists, etc. This would then be used to query PDBs to get symbol information.Crash reportingReporting crashes in some meaningful way. Ideally minidumps would be nice as they could be loaded up and processed in WinDbg. This might be fairly easy as DMPs are just physical memory and processor context, which we already have.Crash deduping / root causingI’ve got some fun techniques for root causing bugs which have been historically successful. I plan to bring those here.Fast resetsBy tracking dirty pages and restoring only modified things we should be able to reset VMs very quickly. This gives us the ability to fuzz at maximum speeds on all cores of a system target. This is similar to what I did in falkervisor so it’s already thought out and designed. It just needs to be ported here.falkervisor modeExtremely fast fuzzing that cancels execution when MMIO or I/O occurs. This allows all the CPU time to be spent in the hypervisor and no emulation time. This has a downside of not supporting things like disk I/O during a fuzz case, but it’s nice.PhilosophySome of the core concepts of this project are absolute minimum modifications to Bochs. This allows us to keep the Bochs portion of this repo up to date.The goal is to also move as much code into Rust and dlls as possible to make the system much more modular and safe. This will hopefully reduce the chances of making silly corruption bugs in the hypervisor itself, causing invalid fuzz results.Currently the hypervisor is a DLL and can be swapped out without changes to Bochs (unless the FFI API changes).Further changes to Bochs itself must be documented clearly, and I’ll be making a document for that shortly to track the changes to Bochs which must be ported and re-evaluated with Bochs updates.Download Applepie

Link: http://feedproxy.google.com/~r/PentestTools/~3/U7xXM25iB_M/applepie-hypervisor-for-fuzzing-built.html