HELK – The Hunting ELK

The Hunting ELK or simply the HELK is one of the first open source hunt platforms with advanced analytics capabilities such as SQL declarative language, graphing, structured streaming, and even machine learning via Jupyter notebooks and Apache Spark over an ELK stack. This project was developed primarily for research, but due to its flexible design and core components, it can be deployed in larger environments with the right configurations and scalable infrastructure.GoalsProvide an open source hunting platform to the community and share the basics of Threat Hunting.Expedite the time it takes to deploy a hunt platform.Improve the testing and development of hunting use cases in an easier and more affordable way.Enable Data Science capabilities while analyzing data via Apache Spark, GraphFrames & Jupyter Notebooks.Current Status: AlphaThe project is currently in an alpha stage, which means that the code and the functionality are still changing. We haven’t yet tested the system with large data sources and in many scenarios. We invite you to try it and welcome any feedback.HELK FeaturesKafka: A distributed publish-subscribe messaging system that is designed to be fast, scalable, fault-tolerant, and durable.Elasticsearch: A highly scalable open-source full-text search and analytics engine.Logstash: A data collection engine with real-time pipelining capabilities.Kibana: An open source analytics and visualization platform designed to work with Elasticsearch.ES-Hadoop: An open-source, stand-alone, self-contained, small library that allows Hadoop jobs (whether using Map/Reduce or libraries built upon it such as Hive, Pig or Cascading or new upcoming libraries like Apache Spark ) to interact with Elasticsearch.Spark: A fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs.GraphFrames: A package for Apache Spark which provides DataFrame-based Graphs.Jupyter Notebook: An open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text.KSQL: Confluent KSQL is the open source, streaming SQL engine that enables real-time data processing against Apache Kafka®. It provides an easy-to-use, yet powerful interactive SQL interface for stream processing on Kafka, without the need to write code in a programming language such as Java or PythonElastalert: ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch Sigma: Sigma is a generic and open signature format that allows you to describe relevant log events in a straightforward manner.Getting StartedWIKIIntroductionArchitecture Overview KafkaLogstashElasticsearchKibanaSparkInstallation(Docker) Accessing the HELK’s ImagesBy default, the HELK’s containers are run in the background (Detached). You can see all your docker containers by running the following command:sudo docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESa97bd895a2b3 cyb3rward0g/helk-spark-worker:2.3.0 “./spark-worker-entr…" About an hour ago Up About an hour 0.0.0.0:8082->8082/tcp helk-spark-worker2cbb31f688e0a cyb3rward0g/helk-spark-worker:2.3.0 "./spark-worker-entr…" About an hour ago Up About an hour 0.0.0.0:8081->8081/tcp helk-spark-worker5d58068aa7e3 cyb3rward0g/helk-kafka-broker:1.1.0 "./kafka-entrypoint.…" About an hour ago Up About an hour 0.0.0.0:9092->9092/tcp helk-kafka-brokerbdb303b09878 cyb3rward0g/helk-kafka-broker:1.1.0 "./kafka-entrypoint.…" About an hour ago Up About an hour 0.0.0.0:9093->9093/tcp helk-kafka-broker27761d1e43d37 cyb3rward0g/helk-nginx:0.0.2 "./nginx-entrypoint.…" About an hour ago Up About an hour 0.0.0.0:80->80/tcp helk-nginxede2a2503030 cyb3rward0g/helk-jupyter:0.32.1 "./jupyter-entrypoin…" About an hour ago Up About an hour 0.0.0.0:4040->4040/tcp, 0.0.0.0:8880->8880/tcp helk-jupyterede19510e959 cyb3rward0g/helk-logstash:6.2.4 "/usr/local/bin/dock…" About an hour ago Up About an hour 5044/tcp, 9600/tcp helk-logstashe92823b24b2d cyb3rward0g/helk-spark-master:2.3.0 "./spark-master-entr…" About an hour ago Up About an hour 0.0.0.0:7077->7077/tcp, 0.0.0.0:8080->8080/tcp helk-spark-master6125921b310d cyb3rward0g/helk-kibana:6.2.4 "./kibana-entrypoint…" About an hour ago Up About an hour 5601/tcp helk-kibana4321d609ae07 cyb3rward0g/helk-zookeeper:3.4.10 "./zookeeper-entrypo…" About an hour ago Up About an hour 2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp helk-zookeeper9cbca145fb3e cyb3rward0g/helk-elasticsearch:6.2.4 "/usr/local/bin/dock…" About an hour ago Up About an hour 9200/tcp, 9300/tcp helk-elasticsearchThen, you will just have to pick which container you want to access and run the following following commands:sudo docker exec -ti bashroot@ede2a2503030:/opt/helk/scripts#ResourcesWelcome to HELK! : Enabling Advanced Analytics CapabilitiesSparkSpark Standalone ModeSetting up a Pentesting.. I mean, a Threat Hunting Lab – Part 5An Integrated API for Mixing Graph and Relational QueriesGraph queries in Spark SQLGraphframes OverviewElastic ProducsElastic SubscriptionsElasticsearch Guidespujadas elk-dockerdeviantony docker-elkAuthorRoberto Rodriguez @Cyb3rWard0g @THE_HELKContributorsJose Luis Rodriguez @Cyb3rPandaHRobby Winchester @robwinchester3Jared Atkinson @jaredatkinsonNate Guagenti @neu5ronLee Christensen @tifkin_ContributingThere are a few things that I would like to accomplish with the HELK as shown in the To-Do list below. I would love to make the HELK a stable build for everyone in the community. If you are interested on making this build a more robust one and adding some cool features to it, PLEASE feel free to submit a pull request. #SharingIsCaringDownload HELK

Link: http://feedproxy.google.com/~r/PentestTools/~3/ZLYzopsUg1Q/helk-hunting-elk.html

MemGuard – Secure Software Enclave For Storage Of Sensitive Information In Memory

Secure software enclave for storage of sensitive information in memory.This package attempts to reduce the likelihood of sensitive data being exposed. It supports all major operating systems and is written in pure Go.FeaturesSensitive data is encrypted and authenticated in memory using xSalsa20 and Poly1305 respectively. The scheme also defends against cold-boot attacks.Memory allocation bypasses the language runtime by using system calls to query the kernel for resources directly. This avoids interference from the garbage-collector.Buffers that store plaintext data are fortified with guard pages and canary values to detect spurious accesses and overflows.Effort is taken to prevent sensitive data from touching the disk. This includes locking memory to prevent swapping and handling core dumps.Kernel-level immutability is implemented so that attempted modification of protected regions results in an access violation.Multiple endpoints provide session purging and safe termination capabilities as well as signal handling to prevent remnant data being left behind.Side-channel attacks are mitigated against by making sure that the copying and comparison of data is done in constant-time.Accidental memory leaks are mitigated against by harnessing the garbage-collector to automatically destroy containers that have become unreachable.Some features were inspired by libsodium, so credits to them.Full documentation and a complete overview of the API can be found here. Interesting and useful code samples can be found within the examples subpackage.Installation$ go get github.com/awnumar/memguardWe strongly encourage you to pin a specific version for a clean and reliable build. This can be accomplished using modules.ContributingUsing the package and identifying points of friction.Reading the source code and looking for improvements.Adding interesting and useful program samples to ./examples.Developing Proof-of-Concept attacks and mitigations.Improving compatibility with more kernels and architectures.Implementing kernel-specific and cpu-specific protections.Writing useful security and crypto libraries that utilise memguard.Submitting performance improvements or benchmarking code.Issues are for reporting bugs and for discussion on proposals. Pull requests should be made against master.Future goalsAbility to stream data to and from encrypted enclave objects.Catch segmentation faults to wipe memory before crashing.Evaluate and improve the strategies in place, particularly for Coffer objects.Formalise a threat model and evaluate our performance in regards to it.Use lessons learned to apply patches upstream to the Go language and runtime.Download Memguard

Link: http://www.kitploit.com/2019/08/memguard-secure-software-enclave-for.html

Dockernymous – A Script Used To Create A Whonix Like Gateway/Workstation Environment With Docker Containers

Dockernymous is a start script for Docker that runs and configures two individual Linux containers in order act as a anonymisation workstation-gateway set up.It’s aimed towards experienced Linux/Docker users, security professionals and penetration testers!The gateway container acts as a Anonymizing Middlebox (see https://trac.torproject.org/projects/tor/wiki/doc/TransparentProxy) and routes ALL traffic from the workstation container through the Tor Network.The idea was to create a whonix-like setup (see https://www.whonix.org) that runs on systems which aren’t able to efficiently run two hardware virtualized machines or don’t have virtualization capacities at all. Requirements:Host (Linux):dockervncviewerxtermcurlGateway Image:Linux (e.g. Alpine, Debian )torprocpsncatiptablesWorkstation Image:Linux (e.g. Kali)‎xfce4 or another desktop environment (for vnc access)tightvncserverInstructions:1. HostTo clone the dockernymous repository type:git clone https://github.com/bcapptain/dockernymous.gitDockernymous needs an up and running Docker environment and a non-default docker network. Let’s create one:docker network create –driver=bridge –subnet=192.168.0.0/24 docker_internal2. Gateway (Alpine):Get a lightweight gateway Image! For example Alpine:docker pull alpineRun the image, update the package list, install iptables & tor:docker run -it alpine /bin/shapk add –update tor iptables iproute2exitFeel free to further customize the gateway for your needs before you extit.To make this permanent you have to create a new image from the gateway container we just set up. Each time you run dockernymous a new container is created from that image and disposed on exit:docker commit [Container ID] my_gatewayGet the container ID by running:docker ps -a3. Workstation (Kali Linux):Get an image for the Workstation. For example, Kali Linux for penetration testing:docker pull kalilinux/kali-linux-dockerUpdate and install the tools you would like to use (see https://www.kali.org/news/kali-linux-metapackages/).docker run -it kalilinux/kali-linux-docker /bin/bashapt-get updateapt-get dist-upgradeapt install kali-linux-top10Make sure the tightvncserver and curl packages are installed which is the case with most Kali Metapackages.apt-get install tightvncserverapt-get install curlInstall xfce4 for a minimal graphical Desktop:$ apt-get install xfce4 $ apt-get clean$ exitAs with the Gateway, to make this permanent you have to create an image from that customized container. Each time you run dockernymous a new container is created and disposed on exit.$ docker commit [Container ID] my_workstationGet the container ID by running:$ docker ps -a4. Run dockernymous In case you changed the names for the images to something different (defaults are: “docker_internal" (network), "my_gateway" (gateway), "my_workstation" (you guess it)) open dockernymous.sh with your favorite editor and update the actual names in the configuration section.Everything should be set up by now, let’s give it a try! Run Dockernymus (don’t forget to ‘cd’ into the cloned folder):bash dockernymous.shor mark it executable once:chmod +x dockernymous.sh and always run it with:./dockernymous.shI’m happy for feedback. Please remember that dockernymous is still under development. The script is pretty messy, yet so consider it as a alpha phased project (no versioning yet).Download Dockernymous

Link: http://feedproxy.google.com/~r/PentestTools/~3/WbwiCRF568Y/dockernymous-script-used-to-create.html

Container Services In Azure, ITProTV – Enterprise Security Weekly #141

    Do you wonder how your team can save costs by lifting and shifting your existing applications to containers, and build micro-services applications to deliver value to your users faster? Use end-to-end developer and CI/CD tools to develop, update, and deploy your containerized applications? Manage containers at scale with a fully managed Kubernetes container […]
The post Container Services In Azure, ITProTV – Enterprise Security Weekly #141 appeared first on Security Weekly.

Link: http://feedproxy.google.com/~r/securityweekly/Lviv/~3/rpojCnz9hl4/

Kubolt – Utility For Scanning Public Kubernetes Clusters

Kubolt is a simple utility for scanning public unauthinticated kubernetes clusters and run commands inside containers.Why?Sometimes, the kubelet port 10250 is open to unauthorized access and makes it possible to run commands inside the containers using getrun function from kubelet:// getRun handles requests to run a command inside a container.func (s *Server) getRun(request *restful.Request, response *restful.Response) { params := getExecRequestParams(request) pod, ok := s.host.GetPodByName(params.podNamespace, params.podName) if !ok { response.WriteError(http.StatusNotFound, fmt.Errorf(“pod does not exist")) return }How?Okay, let’s ask our friend ShodanThe basic query isssl:true port:10250 404Kubelet uses port 10250 with SSL by default, 404 is the HTTP response without URL path.Kubolt asks Shodan by API for list of IP addresses and keeps them for other OSINT actions Firstly, let’s ask Kubelet for running pods and filter hosts where response doesn’t contain Unauthorized and contains container so we can run command inside it.curl -k https://IP-from-Shodan:10250/runningpods/ Anyway, if you find the host without any running pods at the time, keep it for next time when pods might be started You can list all available pods from these requests:curl -k https://IP-from-Shodan:10250/pods/#orcurl http://IP-from-Shodan:10255/pods/ Next kubolt parse response and generate a new request as below:curl -XPOST -k https://IP-from-Shodan:10250/run//<PodName>/<containerName> -d "cmd=<command-to-run>" You can target companies more accurate using Shodan filters such as:asnorgcountrynetInstallmkdir outputpip install -r requirements.txt Runpython kubolt.py –query "asn:123123 org:’ACME Corporation’"#orpython kubolt.py –query "org:’ACME Corporation’ country:UK"ShodanKubolt uses Shodan API and Query Credits accordingly, if you run the tool without query filters then you will probably fire all your creditsImportantThe Tool provided by the author should only be used for educational purposes. The author can not be held responsible for the misuse of the Tool. The author is not responsible for any direct or indirect damage caused due to the usage of the Tool.Download Kubolt

Link: http://feedproxy.google.com/~r/PentestTools/~3/snT7GJXlPRw/kubolt-utility-for-scanning-public.html

Containers and Kubernetes – Application Security Weekly #57

    This last week was pretty busy with announcements and presentations from the Google Next Conference. In 2018 they previewed some security tools and this year many of them are now GA along with a lot of other developer-focused services. Full Show Notes Follow us on Twitter: https://www.twitter.com/securityweekly Hosts
The post Containers and Kubernetes – Application Security Weekly #57 appeared first on Security Weekly.

Link: http://feedproxy.google.com/~r/securityweekly/Lviv/~3/lnbFzMQFpz8/

Docker, ARM, & “Selfie” – Application Security Weekly #56

    In the News segment, The Matrix turns 20, Containers are Weakest Security Leak Again, The Evolution of Application Security in the Serverless World, and more! News Bugs, Breaches, and Bounties! Envoy NULL character injection Envoy path traversal “Selfie” attack in PSK mutual authentication process Facebook app developers leaked millions of user records on […]
The post Docker, ARM, & “Selfie” – Application Security Weekly #56 appeared first on Security Weekly.

Link: http://feedproxy.google.com/~r/securityweekly/Lviv/~3/39u2bPsj8Dk/

Falco, Sysdig – Application Security Weekly #56

    This week, we welcome Loris Degioanni from Sysdig to discuss their open source container native runtime security project called Falco! To learn more about Sysdig, visit: https://securityweekly.com/sysdigFull Show Notes Follow us on Twitter: https://www.twitter.com/securityweekly Announcements Register for our upcoming webcasts with LogRhythm and Recorded Future by going to securityweekly.com/webcasts . If you have […]
The post Falco, Sysdig – Application Security Weekly #56 appeared first on Security Weekly.

Link: http://feedproxy.google.com/~r/securityweekly/Lviv/~3/1liXjiCF9RA/