Search Results for "web crawler source code"
Sort By:
1731 projects for "web crawler source code" with 1 filter applied:
-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.
-
Unimus makes Network Automation and Configuration Management easy.We aim to make automation, disaster recovery, change management and configuration auditing painless and affordable for a network of any size.
-
1
Spatie Crawler
An easy to use, powerful crawler implemented in PHP
Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites. -
2
Heritrix
Internet Archive's open-source, web-scale, web crawler project
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt. -
3
Phoenix Code Editor
Phoenix is a modern open-source Code Editor for the web
Phoenix is a modern open-source and free software code editor for the web, built for the browser. -
4
Screenshot to Code
A neural network that transforms a design mock-up into static websites
Screenshot-to-code is a tool or prototype that attempts to convert UI screenshots (e.g., of mobile or web UIs) into code representations, likely generating layouts, HTML, CSS, or markup from image inputs. It is part of a research/proof-of-concept domain in UI automation and image-to-UI code generation. Mapping visual design to code constructs. Code/UI layout (HTML, CSS, or markup). Examples/demo scripts showing “image UI code”. -
LinkSquares: All-in-One Contract Management PlatformLinkSquares is the leading Contract Lifecycle Management (CLM) software designed to help legal, procurement, and business operations teams master the entire contract lifecycle, from creation to execution and renewal. The platform transforms how companies manage agreements by centralizing data, automating routine work, and providing actionable insights powered by AI. This single, connected source of truth helps teams eliminate manual processes, streamline workflows, boost visibility, and ensure compliance across thousands of contracts, ultimately reducing risk and administrative burden.
-
5
Laravel Web Tinker
Tinker in your browser
Artisan's tinker command is a great way to tinker with your application in the terminal. Unfortunately running a few lines of code, making edits, and copy/pasting code can be bothersome. Wouldn't it be great to tinker in the browser? This package will add a route to your application where you can tinker to your heart's content. In case light hurts your eyes, there's a dark mode too. -
6
whatsapp-web.js
WhatsApp library for NodeJS that connects through the browser app
A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app. Programmatically control WhatsApp whether you're running user or business accounts. It uses Puppeteer to run a real instance of Whatsapp Web to avoid getting blocked. Programmatically control WhatsApp whether you're running user or business accounts. Whatsapp-web.js connects to an official version of WhatsApp Web under the hood, reducing ban risks. The object-oriented approach makes it easy to get... -
7
claude-code-transcripts
Tools for publishing transcripts for Claude Code sessions
claude-code-transcripts is a command-line utility that takes session files exported from Claude Code (in JSON or JSONL format) and turns them into clean, navigable HTML transcripts that can be viewed in any modern web browser. It is designed to make the often dense and verbose outputs from AI coding sessions easier to read, share, and archive by breaking conversations into paginated, annotated pages with navigable timelines of prompts and responses. Users can run this tool locally or fetch... -
8
Proton Web Clients
Monorepo hosting the proton web clients
Proton Web Clients is a monorepo hosting the web applications for Proton’s suite of privacy-focused services, including the core Proton Mail webmail interface and related web apps like Proton Calendar, Proton Drive, Proton Account, Proton VPN, Proton Pass, and other connected tools. It consolidates all web client code, shared modules, dependencies, and development tooling into a single repository, enabling unified maintenance, consistency of design patterns, and efficient evolution of... -
9
X-Crawl
Flexible Node.js AI-assisted crawler library
A high-performance web crawling and scraping framework for Node.js, designed for large-scale data extraction. -
Papirfly: Best user-friendly DAM and Content Creation SoftwarePapirfly provides a single online destination for all your employees and other stakeholders who are engaging with your brand, ensuring consistency in all aspects of their communications. Teams can produce infinite studio-standard marketing materials from bespoke templates, store, share and adapt them for their own markets and stay firmly educated on the brand’s purpose, guidelines and evolution – with no specialist skills or agency help necessary.
-
10
The Apache Struts web framework
Mirror of Apache Struts
The Apache Struts web framework is a free open-source solution for creating Java web applications. Web applications differ from conventional websites in that web applications can create a dynamic response. Many websites deliver only static pages. A web application can interact with databases and business logic engines to customize a response. Web applications based on JavaServer Pages sometimes commingle database code, page design code, and control flow code. ... -
11
Python-Spider
Python3 web crawler practice
Python-Spider is a repository intended to teach or provide examples for writing web spiders / crawlers in Python — part of a broader learning and resource collection by its author. The code and documentation are oriented toward beginners or intermediate learners who want to learn how to fetch, parse, and extract data from websites programmatically. As part of the author’s public learning-path repositories, python-spider likely includes examples of HTTP requests, HTML parsing, maybe... -
12
PaSa
An advanced paper search agent powered by large language models
PaSa is an open-source “paper search agent” built around large language models (LLMs), designed to automate the process of academic literature retrieval with human-like decision making. Instead of simply translating a query into keywords and returning a flat list of matching papers, PaSa uses a dual-agent architecture (Crawler + Selector) that can iteratively search, read, analyze, and filter academic publications — simulating how a researcher might dig through citation networks, expand... -
13
Monaco Editor
A browser based code editor
Monaco Editor is the rich, browser-based code editor that powers Visual Studio Code, providing advanced editing capabilities as a standalone embeddable library for web applications. Models are at the heart of Monaco editor. It's what you interact with when managing content. A model represents a file that has been opened. This could represent a file that exists on a file system, but it doesn't have to. For example, the model holds the text content, determines the language of the content, and... -
14
Llama Coder
Open source Claude Artifacts – built with Llama 3.1 405B
Llama Coder is an open-source tool that lets you generate small applications (often React or web apps) from a single natural-language prompt using the Llama 3 family of models. It’s framed as an open-source “Claude Artifacts”-style experience: you describe the app you want, the tool calls an LLM hosted on Together.ai, and you get back a runnable code artifact. -
15
Happy Coder
Mobile and Web client for Codex and Claude Code, with realtime voice
Happy is an open-source, cross-platform mobile and web client designed to bring powerful AI coding agents such as Claude Code and Codex to your fingertips no matter where you are. At its core, Happy wraps existing AI coding tools with a unified interface, providing real-time voice interactions, encrypted communication, and seamless device switching between desktop and mobile. -
16
OpenAI.fm
Code for openai.fm, a demo for the OpenAI Speech API
OpenAI.fm is an official interactive demo application built to showcase the OpenAI Speech API and its advanced text-to-speech capabilities, providing developers and creators with a hands-on web interface to convert text into high-quality, customizable audio using state-of-the-art TTS models. Developed using Next.js and the OpenAI Speech API, this demo illustrates how the latest neural voice models can produce natural, expressive speech with adjustable styles and voices, highlighting features... -
17
The Vibe Companion
Open-source Claude Code/Codex Web UI
The Companion project by The-Vibe-Company is an open-source web-based UI for interacting with Claude Code, offering a fully graphical interface to launch sessions, stream responses, and approve or deny AI tool usage—all via web browsers on desktop or mobile. Instead of working solely through a command-line interface, developers and creators can start, manage, and visualize Claude Code sessions in a modern web environment built with TypeScript, enabling more natural interaction with AI models. ... -
18
Bloxtor framework
Bloxtor is a professional and intelligent framework
Bloxtor is a professional and intelligent web-development framework designed for No-Coders, Low-Coders, and Coders alike. It functions as a versatile code reader and converter, seamlessly transforming code into visual environments and visual components back into code. The concept is to provide a centralized AI-powered web development platform with multiple levels of Citizen Development, empowering both technical and non-technical developers to easily create any type of web application, while... -
19
Bootstrap
HTML, CSS and JS framework for developing responsive websites and apps
Bootstrap is an HTML, CSS, and JS framework designed for developing responsive, mobile first projects on the web. It's one of the most popular front-end frameworks and for good reason: it simply makes web development faster and easier. People of all skill levels can create projects of all sizes with Bootstrap, and for all types of devices too. With Bootstrap, you get a host of nifty features such as precompiled CSS, impressive scalability, dozens of custom HTML and CSS components and more. -
20
Phaser Editor 2D
A web-based IDE for HTML5 game development. Powered by Phaser
Phaser Editor 2D v3 is a lightweight, multi-platform IDE for HTML5 game development using the Phaser framework. It combines visual tools and code editing capabilities, helping developers design scenes, UI, and game logic in an efficient and organized workflow. -
21
Groq AppGen
Project showcasing Llama 3.3 70B HTML codegen abilities
Groq AppGen is an interactive web application (built with Next.js and TypeScript) that uses Groq’s LLM API to generate or modify web application code based on natural-language prompts. Essentially, you tell the app what kind of web app or page you want (in plain English), and groq-appgen will produce HTML/JSX code scaffolding, layout, and optionally application logic accordingly. It supports iterative feedback: you can refine your prompt, adjust parameters or requirements, and have the app... -
22
Frappe
Low code web framework for real world applications
Frappe is a full-stack, low-code web framework written in Python and JavaScript, used to build scalable and modular enterprise applications. It powers ERPNext and includes tools for REST APIs, user management, document modeling, workflows, and real-time updates. Frappe uses a "model-view-controller" approach with its own ORM and frontend system, enabling rapid development without sacrificing control or performance. -
23
Claude Run
A beautiful web UI for browsing Claude Code conversation history
Claude Run is a user-friendly web interface for browsing, searching, and managing Claude Code conversation history in a visually appealing way, making it easier to revisit past interactions, track project progress, and resume long-running coding sessions. Running locally via a single command (npx claude-run), it launches a web UI that displays all stored Claude Code sessions – including conversation threads, tool calls, and timestamps – in a clean chat layout, helping developers quickly find... -
24
FastAPI Python
FastAPI framework, high performance, easy to learn, fast to code
FastAPI framework, high performance, easy to learn, fast to code, ready for production. FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints. -
25
Prism
Lightweight, robust, elegant syntax highlighting
Prism is a lightweight, extensible syntax highlighter, built with modern web standards in mind. It’s used in millions of websites, including some of those you visit daily.