A team from the Hong Kong University of Science and Technology developed HarnessSQL, a text-to-SQL agent designed to convert natural language into database queries.
The team trained the agent on the open-source Harness framework provided by DeepSeek. To create the DSH-SQL component, the developers adapted DeepSeek Harness into a database agent equipped with four specific tools: listing tables, reading schemas, executing queries, and submitting final answers. The researchers constructed 79 sandboxed databases for testing, converting various DuckDB projects into SQLite format.
During testing, the team ran these databases in a read-only mode with a 60-second timeout, averaging 11,800 tokens across 16 turns and 14 query executions. The authors noted that applying reinforcement learning directly only reached 20.0%, whereas the supervised stage alone yielded 30.4% for the 8B model and 37.8% for the 14B model.
Fine-tuning on trajectories collected without the harness resulted in a low score of 9.6%. The 8B model achieved 27.4% with an 8K window and 45.2% with a 32K window. Among various reinforcement learning algorithms, the team found that GRPO reached 40.7%, GSPO reached 43.7%, and DAPO reached 45.2%.
The project's code is now available on GitHub.
