Omnibus – Open Source Intelligence Collection, Research, And Artifact Management

An Omnibus is defined as a volume containing several novels or other items previously published separately and that is exactly what the InQuest Omnibus project intends to be for Open Source Intelligence collection, research, and artifact management.By providing an easy to use interactive command line application, users are able to create sessions to investigate various artifacts such as IP addresses, domain names, email addresses, usernames, file hashes, Bitcoin addresses, and more as we continue to expand.This project has taken motivation from the greats that came before it such as SpiderFoot, Harpoon, and DataSploit. Much thanks to those great authors for contributing to the world of open source.The application is written with Python 2.7 in mind and has been successfully tested on OSX and Ubuntu 16.04 environments.As this is a pre-release of the final application, there will very likely be some bugs and uncaught exceptions or other weirdness during usage. Though for the most part, it is fully functional and can be used to begin OSINT investigations right away.VocabularyBefore we begin we’ll need to cover some terminology used by Omnibus.Artifact:An item to investigateArtificats can be created in two ways:Using the new command or by being discoverd through module executionSession:Cache of artifacts created after starting the Omnibus CLIEach artifact in a session is given an ID to quickly identify and retrieve the artifact from the cacheCommands can be executed against an artifact either by providing it’s name or it’s corresponding session IDModule:Python script that performs some arbitirary OSINT task against an artifactRunning OmnibusStarting up Omnibus for investigation is a simple as cloning this GitHub repository, installing the Python requirements using pip install -r requirements.txt and running python2.7 omnibus-cli.py. Omnibus Shell – Main StartupFor a visual reference of the CLI, pictured above is the Omnibus console after a new session has been started, 2 artifacts have been added to a session, and the help menu is shown.API KeysYou must set any API keys you’d like to use within modules inside the omnibus/etc/apikeys.json file. This file is a JSON ocument with placeholders for all the services which require API keys, and is only accessed by Omnibus on a per module basis to retrieve the exact API key a module needs to execute.It should be noted that most of the services requiring API keys have free accounts and API keys. Some free accounts may have lower resource limits, but that hasn’t been a problem during smaller daily investigations or testing the application.A handy tip: Use the cat apikeys command to view which keys you do in fact have stored. If modules are failing, check here first to ensure your API key is properly saved.Interactive ConsoleWhen you first run the CLI, you’ll be greeted by a help menu with some basic information. We tried to build the command line script to mimic some common Linux console commands for ease of use. Omnibus provides commands such as cat to show information about an artifact, rm to remove an artifact from the database, ls to view currently session artifacts, and so on.One additional feature of note is the use of the > character for output redirection. For example, if you wish to retrieve the details of an artifact named “inquest.net" saved to a JSON file on your local disk you’d simply run the command: cat inquest.net > inquest-report.json and there it would be! This feature also works with full file paths instead of relative paths.The high level commands you really need to know to use Omnibus are:session start a new sessionnew create a new artifact for investigationmodules display list of available modulesopen <file path> load a text file list of artifacts into Omnibus as artifactscat <artifact name | session id> view beautified JSON database recordsls show all active artifactsrm remove an artifact from the databasewipe clear the current artifact sessionAlso, if you ever need a quick reference on the different commands available for different areas of the application there are sub-help menus for this exact purpose. Using these commands will show you only those commands available relevant to a specific area:general overall commands such as help, history, quit, set, clear, banner, etc.artifacts display commands specific to artifacts and their managementsessions display helpful commands around managing sessionsmodules show a list of all available modulesArtifactsOverviewMost cyber investigations begin with one or more technical indicators, such as an IP address, file hash or email address. After searching and analyzing, relationships begin to form and you can pivot through connected data points. These data points are called Artifacts within Omnibus and represent any item you wish to investigate.Artifacts can be one of the following types:IPv4 addressFQDNEmail AddressBitcoin AddressFile Hash (MD5, SHA1, SHA256, SHA512)User NameCreating & Managing ArtifactsThe command "new" followed by an artifact will create that artifact within your Omnibus session and store a record of the artifact within MongoDB. This record holds the artifact name, type, subtype, module results, source, notes, tags, children information (as needed) and time of creation. Every time you run a module against a created or stored artifact, the database document will be updated to reflect the newly discovered information.To create a new artifact and add it to MongoDB for tracking, run the command new <artifact name>. For example, to start investigation the domain deadbits.org, you would run new deadbits.org.Omnibus will automatically determine what type the artifact is and ensure that only modules for that type are executed against the artifact.When a module is created, new artifacts may be found during the discovery process. For example, running the "dnsresolve" command might find new IPv4 addresses not previously seen by Omnibus. If this is the case, those newly found artifacts are automatically created as new artifacts in Omnibus and linked to their parent with an additional field called "source" to identify from which module they were originally found.Artifacts can be removed from the database using the "delete" command. If you no longer need an artifact, simply run the delete command and specify the artifacts name or the session ID if it has one.SessionsOmnibus makes use of a feature called "sessions". Sessions are temporary caches created via Redis each time you start a CLI session. Every time you create an artifact, that artifacts name is added to the Session along with a numeric key that makes for easy retrieval, searching, and action against the related artifact. For example if you’re session held one item of "inquest.net", instead of needing to execute virustotal inquest.net you could also run virustotal 1 and you would receive the same results. In fact, this works against any module or command that uses an artiface name as it’s first argument.Sessions are here for easy access to artifacts and will be cleared each time you quit the command line session. If you wish to clear the session early, run the command "wipe" and you’ll get a clean slate.Eventually, we would like to add a Cases portion to Omnibus that allows users to create cases of artifacts, move between them, and maintain a more coherent OSINT management platform. Though for this current pre-release, we will be sticking with the Session. 🙂 Interacting with Session IDs instead of Artifact names ModulesOmnibus currently supports the following list of modules. If you have suggestions or modules or would like to write one of your own, please create a pull request.Also, within the Omnibus console, typing the module name will show you the Help information associated with that module.ModulesBlockchain.infoCensysClearBitCymonDNS subdomain enumerationDNS resolutionDShield (SANS ISC)GeoIP lookupFull ContactGist ScrapingGitHub user searchHackedEmails.com email searchHurricane Electric host searchHIBP searchHunter.ioIPInfoIPVoidKeyBaseNmapPassiveTotalPastebinPGP Email and Name lookupRSS Feed ReaderShodanSecurity News ReaderThreatCrowdThreatExpertTotalHashTwitterURLVoidVirusTotalWeb ReconWHOISAs these modules are a work in progress, some may not yet work as expected but this will change over the coming weeks as we hope to officially release version 1.0 to the world!MachinesMachines are a simple way to run all available modules for an artifact type against a given artifact. This is a fast way if you want to gather as much information on a target as possible using a single command.To perform this, simply run the command machine <artifact name|session ID> and wait a few minutes until the modules are finished executing.The only caveat is that this may return a large volume of data and child artifacts depending on the artifact type and the results per module. To remedy this, we are investigating a way to remove specific artifact fields from the stored database document to make it easier for users to prune unwanted data.Quick Reference GuideSome quick commands to remember are:session – start a new artifact cachecat <artifact name>|apikeys – pretty-print an artifacts document or view your stored API keysopen <file path> – load a text file list of artifacts into Omnibus for investigationnew <artifact name> – create a new artifact and add it to MongoDB and your sessionfind <artifact name> – check if an artifact exists in the db and show the resultsReportingReports are the JSON output of an artifacts database document, essentially a text file version of the output of the "cat" command. But by using the report command you may specify an artifact and a filepath you wish to save the output to:omnibus >> report inquest.net /home/adam/intel/osint/reports/inq_report.jsonThis above command overrides the standard report directory of omnibus/reports. By default, and if you do not specify a report path, all reports will be saved to that location. Also, if you do not specify a file name the report will use the following format:[artifact_name]_[timestamp].jsonRedirectionThe output of commands can also be saved to arbitrary text files using the standard Linux character >. For example, if you wish to store the output of a VirusTotal lookup for a host to a file called "vt-lookup.json" you would simply execute:virustotal inquest.net > vt-lookup.jsonBy default the redirected output files are saved in the current working directory, therefore "omnibus/", but if you specify a full path such as virustotal inquest.net > /home/adam/intel/cases/001/vt-lookup.json the JSON formatted output will be saved there.Monitoring ModulesOmnibus will soon be offering the ability to monitor specific keywords and regex patterns across different sources. Once a match is found, an email or text message alert could be sent to the user to inform them on the discovery. This could be leveraged for real-time threat tracking, identifying when threat actors appear on new forums or make a fresh Pastebin post, or simply to stay on top of the current news.Coming monitors include:RSS monitorPastebin monitorGeneric Pastesite monitoringGeneric HTTP/JSON monitoringDownload Omnibus

Link: http://feedproxy.google.com/~r/PentestTools/~3/oqafc7KT-OM/omnibus-open-source-intelligence.html

Otseca – Security Auditing Tool To Search And Dump System Configuration

Otseca is a open source security auditing tool to search and dump system configuration. It allows you to generate reports in HTML or RAW-HTML formats.For more information, see wiki.How To UseIt’s simple:# Clone this repositorygit clone https://github.com/trimstray/otseca# Go into the repositorycd otseca# Install./setup.sh install# Run the appotsecasymlink to bin/otseca is placed in /usr/local/binman page is placed in /usr/local/man/man8RequirementsThis tool working with:GNU/Linux or BSD (testing on Debian, CentOS and FreeBSD)Bash (testing on 4.4.19)Also you will need root access.ReportsOtseca generates reports in html (js, css and other) or raw-html (pure html) formats.Default path for reports is {project}/data/output directory.HTML reports consist of the following blocks:Download Otseca

Link: http://feedproxy.google.com/~r/PentestTools/~3/EUtaEe_vUlo/otseca-security-auditing-tool-to-search.html

Check whether you were hacked in the past

There have been a lot of data breaches over the past few years. We often use the same password on many websites or reuse it after some time. This not only compromises our main social media accounts but also other email accounts.Fol website help us to identify from  email address whether it was part of some data breach or not and help us to patch things up.Mention other websites you know in comment for other people to benefit from.Happy surfing1.    Pwned

Link: http://hackingplayground.blogspot.com/2018/06/check-whether-you-were-hacked-in-past.html

Prowler – Distributed Network Vulnerability Scanner

Prowler is a Network Vulnerability Scanner implemented on a Raspberry Pi Cluster, first developed during Singapore Infosec Community Hackathon – HackSmith v1.0.CapabilitiesScan a network (a particular subnet or a list of IP addresses) for all IP addresses associated with active network devicesDetermine the type of devices using fingerprintingDetermine if there are any open ports on the deviceAssociate the ports with common servicesTest devices against a dictionary of factory default and common credentialsNotify users of security vulnerabilities through an dashboard. Dashboard tourPlanned CapabilitiesGreater variety of vulnerability assessment capabilities (webapp etc.)Select wordlist based on fingerprintHardwareRaspberry Pi Cluster HAT (with 4 * Pi Zero W)Raspberry Pi 3Networking deviceSoftware StackRaspbian Stretch (Controller Pi)Raspbian Stretch Lite (Worker Pi Zero)Note: For ease of setup, use the images provided by Cluster Hat! InstructionsPython 3 (not tested on Python 2)Python packages see requirements.txtAnsible for managing the cluster as a whole (/playbooks)Key Python Packages:dispy (website) is the star of the show. It allows allows us to create a job queue that will be processed by the worker nodes.python-libnmap is the python wrapper around nmap, an open source network scanner. It allows us to scan for open ports on devices.paramiko is a python wrapper around SSH. We use it to probe SSH on devices to test for common credentials.eel is used for the web dashboard (seperate repository, here)rabbitmq (website) is used to pass the results from the cluster to the eel server that is serving the dashboard page.Ansible PlaybooksFor the playbooks to work, ansible must be installed (sudo pip3 install ansible). Configure the IP addresses of the nodes at /etc/ansible/hosts. WARNING: Your mileage may vary as these were only tested on my setupshutdown.yml and reboot.yml self-explanatoryclone_repos.yml clone prowler and dispy repositories (required!) on the worker nodessetup_node.yml installs all required packages on the worker nodes. Does not clone the repositories!Deploying ProwlerClone the git repository: git clone https://github.com/tlkh/prowler.gitInstall dependencies by running sudo pip3 install -r requirements.txt on the controller PiRun ansible-playbook playbooks/setup_node.yml to install the required packages on worker nodes.Clone the prowler and dispy repositories to the worker nodes using ansible-playbook playbooks/clone_repos.ymlRun clusterhat on on the controller Pi to ensure that all Pi Zeros are powered up.Run python3 cluster.py on the controller Pi to start ProwlerTo edit the range of IP addresses being scanned, edit the following lines in cluster.py:test_range = [] for i in range(0, 1): for j in range(100, 200): test_range.append(“172.22." + str(i) + "." + str(j))Old DemosCluster Scan Demonstration Jupyter NotebookSingle Scan Demonstration Jupyter NotebookTry out the web dashboard hereUseful SnippetsTo run ssh command on multiple devices, install pssh and pssh -h pssh-hosts -l username -A -i "command"To create the cluster (in compute.py): cluster = dispy.JobCluster(compute, nodes=’pi0_ip’, ip_addr=’pi3_ip’)Check connectivity: ansible all -m ping or ping p1.local -c 1 && ping p2.local -c 1 && ping p3.local -c 1 && ping p4.local -c 1Temperature Check: /opt/vc/bin/vcgencmd measure_temp && pssh -h workers -l pi -A -i "/opt/vc/bin/vcgencmd measure_temp" | grep temprpimonitor (how to install):Download Prowler

Link: http://feedproxy.google.com/~r/PentestTools/~3/qOTSZ3YjvmY/prowler-distributed-network.html

Z-Wave, SSD, Singapore ISP, and VPN Filter Malware Risk – Hack Naked News #175

This week, net neutrality and what it really means, Qradar vulnerability, trying to secure your mobile device, when Z-Wave attacks, routers are open to attack because of your ISP, Starbucks and XSS, Despacito hackers arrested, rebooting your routers, and more! Daniel Lowrie from ITPro.TV joins us for expert commentary this week, and more on this […]
The post Z-Wave, SSD, Singapore ISP, and VPN Filter Malware Risk – Hack Naked News #175 appeared first on Security Weekly.

Link: http://feedproxy.google.com/~r/securityweekly/Lviv/~3/TKRlvfqru80/

Burpa – A Burp Suite Automation Tool

A Burp Suite Automation Tool With Slack Integration.Requirementsburp-rest-apiBurp Suite ProfessionalslackclientUsage$ python burpa.py -h################################################### __ / /_ __ ___________ ____ _ / __ \/ / / / ___/ __ \/ __ `/ / /_/ / /_/ / / / /_/ / /_/ / /_.___/\__,_/_/ / .___/\__,_/ /_/ burpa version 0.1 / by 0x4D31 ###################################################usage: burpa.py [-h] [-a {scan,proxy-config,stop}] [-pP PROXY_PORT] [-aP API_PORT] [-rT {HTML,XML}] [-r {in-scope,all}] [-sR] [-sAT SLACK_API_TOKEN] [–include-scope [INCLUDE_SCOPE [INCLUDE_SCOPE …]]] [–exclude-scope [EXCLUDE_SCOPE [EXCLUDE_SCOPE …]]] proxy_urlpositional arguments: proxy_url Burp Proxy URLoptional arguments: -h, –help show this help message and exit -a {scan,proxy-config,stop}, –action {scan,proxy-config,stop} -pP PROXY_PORT, –proxy-port PROXY_PORT -aP API_PORT, –api-port API_PORT -rT {HTML,XML}, –report-type {HTML,XML} -r {in-scope,all}, –report {in-scope,all} -sR, –slack-report -sAT SLACK_API_TOKEN, –slack-api-token SLACK_API_TOKEN –include-scope [INCLUDE_SCOPE [INCLUDE_SCOPE …]] –exclude-scope [EXCLUDE_SCOPE [EXCLUDE_SCOPE …]]TEST:$ python burpa.py http://127.0.0.1 –action proxy-config################################################### __ / /_ __ ___________ ____ _ / __ \/ / / / ___/ __ \/ __ `/ / /_/ / /_/ / / / /_/ / /_/ / /_.___/\__,_/_/ / .___/\__,_/ /_/ burpa version 0.1 / by 0x4D31 ###################################################[+] Checking the Burp proxy configuration …[-] Proxy configuration needs to be updated[+] Updating the Burp proxy configuration …[-] Proxy configuration updated$ python burpa.py http://127.0.0.1 –action scan –include-scope http://testasp.vulnweb.com –report in-scope –slack-report################################################### __ / /_ __ ___________ ____ _ / __ \/ / / / ___/ __ \/ __ `/ / /_/ / /_/ / / / /_/ / /_/ / /_.___/\__,_/_/ / .___/\__,_/ /_/ burpa version 0.1 / by 0x4D31 ###################################################[+] Retrieving the Burp proxy history …[-] Found 4 unique targets in proxy history[+] Updating the scope …[-] http://testasp.vulnweb.com included in scope[+] Active scan started …[-] http://testasp.vulnweb.com Added to the scan queue[-] Scan in progress: %100[+] Scan completed[+] Scan issues for http://testasp.vulnweb.com: – Issue: Robots.txt file, Severity: Information – Issue: Cross-domain Referer leakage, Severity: Information – Issue: Cleartext submission of password, Severity: High – Issue: Frameable response (potential Clickjacking), Severity: Information – Issue: Password field with autocomplete enabled, Severity: Low – Issue: Cross-site scripting (reflected), Severity: High – Issue: Unencrypted communications, Severity: Low – Issue: Path-relative style sheet import, Severity: Information – Issue: Cookie without HttpOnly flag set, Severity: Low – Issue: File path traversal, Severity: High – Issue: SQL injection, Severity: High[+] Downloading HTML/XML report for http://testasp.vulnweb.com[-] Scan report saved to /tmp/burp-report_20170807-235135_http-testasp.vulnweb.com.html[+] Burp scan report uploaded to SlackDownload Burpa

Link: http://feedproxy.google.com/~r/PentestTools/~3/rH37EgRftO0/burpa-burp-suite-automation-tool.html

Salt-Scanner – Linux Vulnerability Scanner Based On Salt Open And Vulners Audit API

A linux vulnerability scanner based on Vulners Audit API and Salt Open, with Slack notifications and JIRA integration.FeaturesSlack notification and report uploadJIRA integrationOpsGenie integrationRequirementsSalt Open 2016.11.x (salt-master, salt-minion)¹Python 2.7salt (you may need to install gcc, gcc-c++, python dev)slackclientjiraopsgenie-sdkNote: Salt Master and Minion versions should match. Salt-Scanner supports Salt version 2016.11.x. if you are using version 2017.7.x, replace “expr_form" with "tgt_type" in salt-scanner.py.Usage$ ./salt-scanner.py -h ========================================================== Vulnerability scanner based on Vulners API and Salt Open _____ _ _ _____ / ___| | | | / ___| \ `–. __ _| | |_ \ `–. ___ __ _ _ __ _ __ ___ _ __ `–. \/ _` | | __| `–. \/ __/ _` | ‘_ \| ‘_ \ / _ \ ‘__|/\__/ / (_| | | |_ /\__/ / (_| (_| | | | | | | | __/ | \____/ \__,_|_|\__| \____/ \___\__,_|_| |_|_| |_|\___|_| Salt-Scanner 0.1 / by 0x4D31 ==========================================================usage: salt-scanner.py [-h] [-t TARGET_HOSTS] [-tF {glob,list,grain}] [-oN OS_NAME] [-oV OS_VERSION]optional arguments: -h, –help show this help message and exit -t TARGET_HOSTS, –target-hosts TARGET_HOSTS -tF {glob,list,grain}, –target-form {glob,list,grain} -oN OS_NAME, –os-name OS_NAME -oV OS_VERSION, –os-version OS_VERSION$ sudo SLACK_API_TOKEN="EXAMPLETOKEN" ./salt-scanner.py -t "*" ========================================================== Vulnerability scanner based on Vulners API and Salt Open _____ _ _ _____ / ___| | | | / ___| \ `–. __ _| | |_ \ `–. ___ __ _ _ __ _ __ ___ _ __ `–. \/ _` | | __| `–. \/ __/ _` | ‘_ \| ‘_ \ / _ \ ‘__|/\__/ / (_| | | |_ /\__/ / (_| (_| | | | | | | | __/ | \____/ \__,_|_|\__| \____/ \___\__,_|_| |_|_| |_|\___|_| Salt-Scanner 0.1 / by 0x4D31 ==========================================================+ No default OS is configured. Detecting OS…+ Detected Operating Systems: – OS Name: centos, OS Version: 7+ Getting the Installed Packages…+ Started Scanning ‘10.10.10.55’… – Total Packages: 357 – 6 Vulnerable Packages Found – Severity: Low+ Started Scanning ‘10.10.10.56’… – Total Packages: 392 – 6 Vulnerable Packages Found – Severity: Critical+ Finished scanning 2 host (target hosts: ‘*’).2 Hosts are vulnerable!+ Output file created: 20170622-093138_232826a7-983f-499b-ad96-7b8f1a75c1d7.txt+ Full report uploaded to Slack+ JIRA Issue created: VM-16+ OpsGenie alert createdYou can also use Salt Grains such as ec2_tags in target_hosts:$ sudo ./salt-scanner.py –target-hosts "ec2_tags:Role:webapp" –target-form grainSlack AlertDownload Salt-Scanner

Link: http://feedproxy.google.com/~r/PentestTools/~3/Ox5vp0e8ctQ/salt-scanner-linux-vulnerability.html

3 Best Wireshark Alternatives for Android

Wireshark is the most popular network packet analyser that lets you see network traffic going out and coming in to all computers in the network. So, you can see anything on your network that’s not encrypted. The only problem is that Wireshark is not available for Android. While most of the people now prefer Android […]
The post 3 Best Wireshark Alternatives for Android appeared first on UseThisTip.

Link: http://feedproxy.google.com/~r/blogspot/csAFg/~3/zFUke8qjSGY/best-wireshark-alternatives-for-android.html