{"name":"Subwiz: A Lightweight GPT Model for Subdomain Discovery","description":"Subwiz is an innovative, lightweight GPT model designed specifically for discovering subdomains. It leverages a transformer architecture, trained on extensive subdomain lists, to predict new subdomains efficiently. This tool is ideal for security researchers and developers looking to expand their subdomain enumeration capabilities.","github":"https://github.com/hadriansecurity/subwiz","url":"https://osrepos.com/repo/hadriansecurity-subwiz","source":"osrepos.com","sourceDescription":"This repository profile is provided by osrepos.com, an open source repository discovery platform.","repositoryProfile":"https://osrepos.com/repo/hadriansecurity-subwiz","generatedFor":"open source discovery and AI-assisted research","markdown":"https://osrepos.com/repo/hadriansecurity-subwiz.md","json":"https://osrepos.com/repo/hadriansecurity-subwiz.json","topics":["Python","Subdomain Enumeration","GPT Model","Cybersecurity","OSINT","Security Tools","Machine Learning"],"keywords":["Python","Subdomain Enumeration","GPT Model","Cybersecurity","OSINT","Security Tools","Machine Learning"],"stars":null,"summary":"Subwiz is an innovative, lightweight GPT model designed specifically for discovering subdomains. It leverages a transformer architecture, trained on extensive subdomain lists, to predict new subdomains efficiently. This tool is ideal for security researchers and developers looking to expand their subdomain enumeration capabilities.","content":"## Introduction\n\nSubwiz is a powerful, yet lightweight, GPT model meticulously trained to discover subdomains. Developed by Hadrian Security, this tool provides an advanced approach to subdomain enumeration, utilizing a transformer architecture to predict potential subdomains based on known inputs. It's a valuable asset for cybersecurity professionals and researchers aiming to enhance their reconnaissance efforts.\n\n## Installation\n\nInstalling Subwiz is straightforward, allowing you to quickly integrate it into your workflow. You can install it using `pipx` or `pip`:\n\nbash\npipx install subwiz\n\n\nOR\n\nbash\npip install subwiz\n\n\n## Examples\n\nSubwiz is designed to work effectively, often in conjunction with other tools, or as a standalone Python library.\n\n### Recommended Command-Line Use\n\nFor optimal results, it's recommended to first use a tool like [Subfinder](https://github.com/projectdiscovery/subfinder){:target=\"_blank\"} to gather subdomains from passive sources, then feed those into Subwiz:\n\nbash\nsubfinder -d example.com -o subdomains.txt\nsubwiz -i subdomains.txt\n\n\n### Using Subwiz in Python\n\nSubwiz can also be seamlessly integrated into your Python scripts, offering programmatic control over its subdomain discovery capabilities:\n\npython\nimport subwiz\n\nknown_subdomains = ['test1.example.com', 'test2.example.com']\nnew_subdomains = subwiz.run(input_domains=known_subdomains)\nprint(new_subdomains)\n\n\n### Supported Switches\n\nSubwiz offers various command-line options to fine-tune its behavior:\n\ncommandline\nusage: cli.py [-h] -i INPUT_FILE [-o OUTPUT_FILE] [-n NUM_PREDICTIONS] [--no-resolve]\n              [--force-download] [--max-recursion MAX_RECURSION] [-t TEMPERATURE]\n              [-d {auto,cpu,cuda,mps}] [-m MAX_NEW_TOKENS]\n              [--resolution-concurrency RESOLUTION_CONCURRENCY] [--multi-apex] [-q] [-s]\n\noptions:\n  -h, --help            show this help message and exit\n  -i, --input-file INPUT_FILE\n                        file containing new-line-separated subdomains. (default: None)\n  -o, --output-file OUTPUT_FILE\n                        output file to write new-line separated subdomains to. (default: None)\n  -n, --num-predictions NUM_PREDICTIONS\n                        number of subdomains to predict. (default: 500)\n  --no-resolve          do not resolve the output subdomains. (default: False)\n  --force-download      download model and tokenizer files, even if cached. (default: False)\n  --max-recursion MAX_RECURSION\n                        maximum number of times the inference process will recursively re-run\n                        after finding new subdomains. (default: 5)\n  -t, --temperature TEMPERATURE\n                        add randomness to the model (recommended ? 0.3). (default: 0.0)\n  -d, --device {auto,cpu,cuda,mps}\n                        hardware to run the transformer model on. (default: auto)\n  -m, --max-new-tokens MAX_NEW_TOKENS\n                        maximum length of predicted subdomains in tokens. (default: 10)\n  --resolution-concurrency RESOLUTION_CONCURRENCY\n                        number of concurrent resolutions. (default: 128)\n  --multi-apex          allow multiple apex domains in the input file. runs inference for each\n                        apex separately. (default: False)\n  -q, --quiet           useful for piping into another tool. (default: False)\n  -s, --silent          do not print any output. requires --output-file. (default: False)\n\n\n## Why Use Subwiz\n\nSubwiz stands out due to its innovative application of a lightweight GPT model for subdomain discovery. Unlike traditional brute-forcing or dictionary-based methods, Subwiz learns patterns from existing subdomains to intelligently predict new ones, potentially uncovering subdomains that other tools might miss.\n\n### Model Architecture\n\nSubwiz is built upon an ultra-lightweight transformer model, inspired by [nanoGPT](https://github.com/karpathy/nanoGPT/tree/master){:target=\"_blank\"}. It boasts 17.3 million parameters and was trained on 26 million tokens, derived from extensive lists of subdomains from passive sources. Its tokenizer was also trained on these same lists, comprising 8192 tokens.\n\n### Intelligent Inference\n\nWhile many generative transformer models predict a single output sequence, Subwiz employs a beam search algorithm to predict the N most likely sequences. This approach enhances its ability to generate a diverse and relevant set of potential subdomains, making it a highly effective tool for comprehensive enumeration.\n\n### Hugging Face Integration\n\nThe Subwiz model is conveniently hosted on Hugging Face as [HadrianSecurity/subwiz](https://huggingface.co/HadrianSecurity/subwiz){:target=\"_blank\"}. The model and tokenizer files are automatically downloaded the first time you run Subwiz, ensuring a smooth setup process.\n\n## Links\n\n*   **GitHub Repository:** [hadriansecurity/subwiz](https://github.com/hadriansecurity/subwiz){:target=\"_blank\"}\n*   **Hugging Face Model:** [HadrianSecurity/subwiz](https://huggingface.co/HadrianSecurity/subwiz){:target=\"_blank\"}","metrics":{"detailViews":8,"githubClicks":13},"dates":{"published":null,"modified":"2025-11-27T08:00:59.000Z"}}