准备创建mysqlv1分支的提交
This commit is contained in:
83
IMPORT_README.md
Normal file
83
IMPORT_README.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# 邮箱导入工具使用说明
|
||||
|
||||
这个导入工具用于将邮箱账号导入到MySQL数据库中,支持Redis缓存。
|
||||
|
||||
## 功能特点
|
||||
|
||||
1. 支持MySQL数据库存储
|
||||
2. 可选启用Redis缓存
|
||||
3. 详细的导入日志
|
||||
4. 自动处理重复邮箱
|
||||
5. Windows平台兼容性优化
|
||||
|
||||
## 前置条件
|
||||
|
||||
1. MySQL/MariaDB数据库服务已运行
|
||||
2. 已在`config.yaml`中配置好数据库连接信息
|
||||
3. Redis服务(可选)
|
||||
|
||||
## 邮箱数据格式
|
||||
|
||||
邮箱数据文件应使用以下格式,每行一个账号:
|
||||
|
||||
```
|
||||
email@example.com----密码----client_id----refresh_token
|
||||
```
|
||||
|
||||
字段说明:
|
||||
- `email`: 邮箱地址
|
||||
- `password`: 邮箱密码
|
||||
- `client_id`: Microsoft应用的客户端ID
|
||||
- `refresh_token`: Microsoft OAuth的刷新令牌
|
||||
|
||||
## 使用方法
|
||||
|
||||
1. 确保MySQL数据库已正确配置
|
||||
|
||||
编辑`config.yaml`文件,设置正确的MySQL连接信息:
|
||||
```yaml
|
||||
database:
|
||||
host: "localhost"
|
||||
port: 3306
|
||||
username: "auto_cursor_reg"
|
||||
password: "your_password"
|
||||
database: "auto_cursor_reg"
|
||||
```
|
||||
|
||||
2. 准备邮箱数据文件
|
||||
|
||||
默认读取`email.txt`文件,也可以在`config.yaml`中指定:
|
||||
```yaml
|
||||
email:
|
||||
file_path: "path/to/your/email_file.txt"
|
||||
```
|
||||
|
||||
3. 运行导入工具
|
||||
|
||||
```bash
|
||||
python import_emails.py
|
||||
```
|
||||
|
||||
4. 查看导入结果
|
||||
|
||||
导入过程和结果会显示在控制台,详细日志保存在`import_emails.log`文件中。
|
||||
|
||||
## 常见问题
|
||||
|
||||
1. **无法连接数据库**
|
||||
- 检查MySQL服务是否启动
|
||||
- 确认用户名和密码正确
|
||||
- 确认数据库名称存在
|
||||
|
||||
2. **导入失败**
|
||||
- 检查邮箱数据文件格式是否正确
|
||||
- 查看导入日志获取详细错误信息
|
||||
|
||||
3. **重复邮箱处理**
|
||||
- 系统会自动跳过重复的邮箱,并在日志中标记
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 导入前建议备份原有数据
|
||||
- 大批量导入时,建议适当增加MySQL的连接超时设置
|
||||
- 导入成功后可以运行主程序开始注册流程
|
||||
91
INIT_README.md
Normal file
91
INIT_README.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# 数据库初始化工具使用说明
|
||||
|
||||
这个初始化工具用于自动配置MySQL数据库和Redis(可选),创建所需的表结构,并更新配置文件。此工具适用于服务器端首次部署时的快速配置。
|
||||
|
||||
## 功能特点
|
||||
|
||||
1. 交互式配置MySQL数据库和用户
|
||||
2. 自动创建所需的表结构
|
||||
3. 可选配置Redis缓存
|
||||
4. 自动更新config.yaml配置文件
|
||||
5. 详细的操作日志
|
||||
|
||||
## 前置条件
|
||||
|
||||
1. 已安装MySQL/MariaDB服务
|
||||
2. 已安装Redis服务(可选)
|
||||
3. 知道MySQL root用户密码
|
||||
4. 安装必要的Python依赖:`pip install -r requirements.txt`
|
||||
|
||||
## 使用方法
|
||||
|
||||
1. 安装必要的依赖
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
2. 运行初始化脚本
|
||||
|
||||
```bash
|
||||
python init_database.py
|
||||
```
|
||||
|
||||
3. 根据提示输入信息
|
||||
- MySQL root用户名和密码
|
||||
- 应用程序数据库和用户设置
|
||||
- Redis配置(可选)
|
||||
|
||||
4. 初始化完成后,脚本会:
|
||||
- 创建数据库和用户
|
||||
- 设置适当的权限
|
||||
- 创建必要的表结构
|
||||
- 更新配置文件
|
||||
|
||||
## 配置选项说明
|
||||
|
||||
### MySQL配置
|
||||
- **主机地址**: MySQL服务器地址,默认为`localhost`
|
||||
- **端口**: MySQL服务端口,默认为`3306`
|
||||
- **Root用户**: 有权限创建数据库和用户的MySQL管理员账号
|
||||
- **数据库名**: 应用程序使用的数据库名,默认为`auto_cursor_reg`
|
||||
- **应用用户名**: 应用程序使用的数据库用户,默认为`auto_cursor_reg`
|
||||
|
||||
### Redis配置
|
||||
- **是否启用**: 是否使用Redis缓存
|
||||
- **主机地址**: Redis服务器地址,默认为`127.0.0.1`
|
||||
- **端口**: Redis服务端口,默认为`6379`
|
||||
- **密码**: Redis认证密码(如果设置了)
|
||||
- **数据库索引**: Redis数据库索引,默认为`0`
|
||||
|
||||
## 常见问题
|
||||
|
||||
1. **无法连接到MySQL**
|
||||
- 确认MySQL服务已启动
|
||||
- 验证root密码是否正确
|
||||
- 检查防火墙设置
|
||||
|
||||
2. **无法连接到Redis**
|
||||
- 确认Redis服务已启动
|
||||
- 验证Redis密码是否正确
|
||||
- 如不需要Redis可选择禁用
|
||||
|
||||
3. **权限问题**
|
||||
- 确保使用的用户有创建数据库和用户的权限
|
||||
|
||||
4. **配置文件备份**
|
||||
- 脚本会自动备份原始配置文件为`config.yaml.bak`
|
||||
|
||||
## 完成后的步骤
|
||||
|
||||
初始化完成后,您可以:
|
||||
|
||||
1. 导入邮箱账号:
|
||||
```bash
|
||||
python import_emails.py
|
||||
```
|
||||
|
||||
2. 运行主程序:
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
107
MYSQL_README.md
Normal file
107
MYSQL_README.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Cursor注册工具 - MySQL & Redis支持
|
||||
|
||||
本文档说明了如何将Cursor注册工具从SQLite数据库迁移到MySQL数据库,并可选地启用Redis缓存以提高性能。
|
||||
|
||||
## 变更内容
|
||||
|
||||
1. 数据库后端从SQLite更换为MySQL
|
||||
2. 可选启用Redis缓存
|
||||
3. 提供数据迁移脚本
|
||||
4. 优化数据库查询性能
|
||||
5. 增加数据库连接池管理
|
||||
|
||||
## 前置要求
|
||||
|
||||
1. Python 3.7+
|
||||
2. MySQL/MariaDB 服务器
|
||||
3. Redis服务器 (可选)
|
||||
4. 安装依赖:`pip install -r requirements.txt`
|
||||
|
||||
## 配置说明
|
||||
|
||||
在`config.yaml`中添加了MySQL和Redis相关配置:
|
||||
|
||||
```yaml
|
||||
# 数据库配置
|
||||
database:
|
||||
# SQLite配置(兼容旧版本)
|
||||
path: "cursor.db"
|
||||
pool_size: 10
|
||||
|
||||
# MySQL配置
|
||||
host: "localhost"
|
||||
port: 3306
|
||||
username: "root"
|
||||
password: ""
|
||||
database: "cursor_register"
|
||||
|
||||
# 是否使用Redis缓存
|
||||
use_redis: true
|
||||
|
||||
# Redis配置(可选,当use_redis为true时生效)
|
||||
redis:
|
||||
host: "127.0.0.1"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
```
|
||||
|
||||
## 数据迁移步骤
|
||||
|
||||
1. 确保MySQL服务器已启动,并已创建好数据库
|
||||
2. 更新`config.yaml`配置文件,设置正确的数据库连接信息
|
||||
3. 运行迁移脚本:`python migrate_db.py`
|
||||
4. 迁移脚本会自动创建表结构并将旧数据导入MySQL
|
||||
|
||||
## 使用Redis缓存
|
||||
|
||||
若要启用Redis缓存以提高性能:
|
||||
|
||||
1. 安装Redis服务器:
|
||||
- Windows: 使用WSL或[Windows版Redis](https://github.com/microsoftarchive/redis/releases)
|
||||
- Linux: `sudo apt install redis-server` (Ubuntu) 或 `sudo yum install redis` (CentOS)
|
||||
- macOS: `brew install redis`
|
||||
|
||||
2. 在`config.yaml`中设置`use_redis: true`并配置Redis连接信息
|
||||
3. 确保安装了`aioredis`包:`pip install aioredis>=2.0.0`
|
||||
|
||||
## 常见问题
|
||||
|
||||
1. **无法连接到MySQL**
|
||||
- 确认MySQL服务已启动
|
||||
- 检查用户名和密码是否正确
|
||||
- 确认数据库是否已创建
|
||||
- 检查防火墙设置
|
||||
|
||||
2. **无法连接到Redis**
|
||||
- 确认Redis服务已启动
|
||||
- 检查端口和密码设置
|
||||
- 如不需要Redis,可设置`use_redis: false`
|
||||
|
||||
3. **迁移失败**
|
||||
- 检查原SQLite数据库文件是否存在且有效
|
||||
- 确认MySQL用户有创建表和写入数据的权限
|
||||
|
||||
## 性能调优
|
||||
|
||||
1. 优化MySQL配置:
|
||||
```ini
|
||||
[mysqld]
|
||||
innodb_buffer_pool_size = 128M
|
||||
innodb_log_file_size = 32M
|
||||
max_connections = 100
|
||||
```
|
||||
|
||||
2. 优化Redis配置:
|
||||
```
|
||||
maxmemory 128mb
|
||||
maxmemory-policy allkeys-lru
|
||||
```
|
||||
|
||||
3. 优化连接池大小:根据并发需要调整`pool_size`参数
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 请确保定期备份MySQL数据库
|
||||
- Redis仅用于缓存,断电或重启会丢失缓存数据
|
||||
- 如在多机部署,需确保时区配置一致
|
||||
23
config.yaml
23
config.yaml
@@ -6,12 +6,31 @@ global:
|
||||
|
||||
# 数据库配置
|
||||
database:
|
||||
# SQLite配置(兼容旧版本)
|
||||
path: "cursor.db"
|
||||
pool_size: 10
|
||||
|
||||
# MySQL配置
|
||||
host: "localhost"
|
||||
port: 3306
|
||||
username: "auto_cursor_reg"
|
||||
password: "this_password_jiaqiao"
|
||||
database: "auto_cursor_reg"
|
||||
|
||||
# 是否使用Redis缓存
|
||||
# 如果使用Python 3.12,请确保安装redis>=4.2.0而不是aioredis
|
||||
use_redis: true
|
||||
|
||||
# Redis配置(可选,当use_redis为true时生效)
|
||||
redis:
|
||||
host: "127.0.0.1"
|
||||
port: 6379
|
||||
password: ""
|
||||
db: 0
|
||||
|
||||
# 代理配置
|
||||
proxy:
|
||||
api_url: "https://api.proxy.com/getProxy"
|
||||
api_url: "https://share.proxy.qg.net/get?key=969331C5&num=1&area=&isp=0&format=txt&seq=\r\n&distinct=false"
|
||||
batch_size: 100
|
||||
check_interval: 300
|
||||
|
||||
@@ -28,7 +47,7 @@ email:
|
||||
captcha:
|
||||
provider: "capsolver" # 可选值: "capsolver" 或 "yescaptcha"
|
||||
capsolver:
|
||||
api_key: "CAP-E0A11882290AC7ADE2F799286B8E2DA497D7CD0510BFA477F3900507809F8AA3"
|
||||
api_key: "CAP-36D01B0995C7C8705DF68ACCFE4E2004FE182DDA72AC5A80F25F1E3B601C31F0"
|
||||
website_url: "https://authenticator.cursor.sh"
|
||||
website_key: "0x4AAAAAAAMNIvC45A4Wjjln"
|
||||
yescaptcha:
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from dataclasses import dataclass
|
||||
from typing import Tuple
|
||||
from typing import Tuple, Optional
|
||||
|
||||
import yaml
|
||||
|
||||
@@ -13,8 +13,25 @@ class GlobalConfig:
|
||||
|
||||
@dataclass
|
||||
class DatabaseConfig:
|
||||
path: str
|
||||
pool_size: int
|
||||
# SQLite的配置字段保留,用于兼容
|
||||
path: Optional[str] = None
|
||||
pool_size: int = 10
|
||||
# MySQL配置
|
||||
host: str = "localhost"
|
||||
port: int = 3306
|
||||
username: str = "auto_cursor_reg"
|
||||
password: str = "this_password_jiaqiao"
|
||||
database: str = "auto_cursor_reg"
|
||||
# Redis配置
|
||||
use_redis: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class RedisConfig:
|
||||
host: str
|
||||
port: int
|
||||
password: str = ""
|
||||
db: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -61,29 +78,47 @@ class CaptchaConfig:
|
||||
class Config:
|
||||
global_config: GlobalConfig
|
||||
database_config: DatabaseConfig
|
||||
proxy_config: ProxyConfig
|
||||
register_config: RegisterConfig
|
||||
email_config: EmailConfig
|
||||
captcha_config: CaptchaConfig
|
||||
redis_config: Optional[RedisConfig] = None
|
||||
proxy_config: ProxyConfig = None
|
||||
register_config: RegisterConfig = None
|
||||
email_config: EmailConfig = None
|
||||
captcha_config: CaptchaConfig = None
|
||||
|
||||
@classmethod
|
||||
def from_yaml(cls, path: str = "config.yaml"):
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
data = yaml.safe_load(f)
|
||||
|
||||
# 创建 database 配置对象
|
||||
db_config = DatabaseConfig(**data.get('database', {}))
|
||||
|
||||
# 创建 redis 配置对象(如果有)
|
||||
redis_config = None
|
||||
if 'redis' in data and db_config.use_redis:
|
||||
redis_config = RedisConfig(**data['redis'])
|
||||
|
||||
# 创建 captcha 配置对象
|
||||
captcha_data = data['captcha']
|
||||
captcha_config = CaptchaConfig(
|
||||
provider=captcha_data['provider'],
|
||||
capsolver=CapsolverConfig(**captcha_data['capsolver']),
|
||||
yescaptcha=YesCaptchaConfig(**captcha_data['yescaptcha'])
|
||||
)
|
||||
captcha_data = data.get('captcha', {})
|
||||
captcha_config = None
|
||||
if captcha_data:
|
||||
captcha_config = CaptchaConfig(
|
||||
provider=captcha_data.get('provider', 'capsolver'),
|
||||
capsolver=CapsolverConfig(**captcha_data.get('capsolver', {})),
|
||||
yescaptcha=YesCaptchaConfig(**captcha_data.get('yescaptcha', {}))
|
||||
)
|
||||
|
||||
# 创建其他配置对象
|
||||
global_config = GlobalConfig(**data.get('global', {}))
|
||||
proxy_config = ProxyConfig(**data.get('proxy', {})) if 'proxy' in data else None
|
||||
register_config = RegisterConfig(**data.get('register', {})) if 'register' in data else None
|
||||
email_config = EmailConfig(**data.get('email', {})) if 'email' in data else None
|
||||
|
||||
return cls(
|
||||
global_config=GlobalConfig(**data['global']),
|
||||
database_config=DatabaseConfig(**data['database']),
|
||||
proxy_config=ProxyConfig(**data['proxy']),
|
||||
register_config=RegisterConfig(**data['register']),
|
||||
email_config=EmailConfig(**data['email']),
|
||||
global_config=global_config,
|
||||
database_config=db_config,
|
||||
redis_config=redis_config,
|
||||
proxy_config=proxy_config,
|
||||
register_config=register_config,
|
||||
email_config=email_config,
|
||||
captcha_config=captcha_config
|
||||
)
|
||||
|
||||
300
core/database.py
300
core/database.py
@@ -1,86 +1,274 @@
|
||||
import asyncio
|
||||
import json
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any, List, Optional
|
||||
from typing import Any, Dict, List, Optional, Tuple, Union
|
||||
|
||||
import aiosqlite
|
||||
import aiomysql
|
||||
from loguru import logger
|
||||
|
||||
# 使用条件导入替代直接导入
|
||||
REDIS_AVAILABLE = False
|
||||
try:
|
||||
# 尝试导入redis.asyncio (Redis-py 4.2.0+)
|
||||
import redis.asyncio as redis_asyncio
|
||||
REDIS_AVAILABLE = True
|
||||
REDIS_TYPE = "redis-py"
|
||||
except ImportError:
|
||||
try:
|
||||
# 尝试导入aioredis (旧版本)
|
||||
import aioredis
|
||||
REDIS_AVAILABLE = True
|
||||
REDIS_TYPE = "aioredis"
|
||||
except (ImportError, TypeError):
|
||||
REDIS_AVAILABLE = False
|
||||
REDIS_TYPE = None
|
||||
|
||||
from core.config import Config
|
||||
|
||||
|
||||
class DatabaseManager:
|
||||
def __init__(self, config: Config):
|
||||
self.db_path = config.database_config.path
|
||||
self._pool_size = config.database_config.pool_size
|
||||
self._pool: List[aiosqlite.Connection] = []
|
||||
# 数据库配置
|
||||
self.db_config = config.database_config
|
||||
self._pool_size = self.db_config.pool_size
|
||||
self._pool = None # 连接池
|
||||
self._pool_lock = asyncio.Lock()
|
||||
|
||||
# Redis配置
|
||||
self.use_redis = self.db_config.use_redis
|
||||
self.redis_config = config.redis_config if hasattr(config, 'redis_config') else None
|
||||
self.redis = None
|
||||
|
||||
async def initialize(self):
|
||||
"""初始化数据库连接池"""
|
||||
logger.info("初始化数据库连接池")
|
||||
async with aiosqlite.connect(self.db_path) as db:
|
||||
await db.execute('''
|
||||
CREATE TABLE IF NOT EXISTS email_accounts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
email TEXT UNIQUE NOT NULL,
|
||||
password TEXT NOT NULL,
|
||||
client_id TEXT NOT NULL,
|
||||
refresh_token TEXT NOT NULL,
|
||||
in_use BOOLEAN DEFAULT 0,
|
||||
cursor_password TEXT,
|
||||
cursor_cookie TEXT,
|
||||
sold BOOLEAN DEFAULT 0,
|
||||
status TEXT DEFAULT 'pending',
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
)
|
||||
''')
|
||||
await db.commit()
|
||||
|
||||
# 初始化连接池
|
||||
for i in range(self._pool_size):
|
||||
conn = await aiosqlite.connect(self.db_path)
|
||||
self._pool.append(conn)
|
||||
|
||||
# 创建MySQL连接池
|
||||
try:
|
||||
logger.info(f"连接MySQL: {self.db_config.host}:{self.db_config.port}, 用户: {self.db_config.username}, 数据库: {self.db_config.database}")
|
||||
self._pool = await aiomysql.create_pool(
|
||||
host=self.db_config.host,
|
||||
port=self.db_config.port,
|
||||
user=self.db_config.username,
|
||||
password=self.db_config.password,
|
||||
db=self.db_config.database,
|
||||
maxsize=self._pool_size,
|
||||
autocommit=True,
|
||||
charset='utf8mb4'
|
||||
)
|
||||
logger.info("MySQL连接池创建成功")
|
||||
except Exception as e:
|
||||
logger.error(f"MySQL连接池创建失败: {str(e)}")
|
||||
logger.error("请检查MySQL配置是否正确,以及MySQL服务是否已启动")
|
||||
logger.info(f"您可能需要创建MySQL用户和数据库:")
|
||||
logger.info(f" CREATE USER '{self.db_config.username}'@'localhost' IDENTIFIED BY '{self.db_config.password}';")
|
||||
logger.info(f" CREATE DATABASE {self.db_config.database} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;")
|
||||
logger.info(f" GRANT ALL PRIVILEGES ON {self.db_config.database}.* TO '{self.db_config.username}'@'localhost';")
|
||||
logger.info(f" FLUSH PRIVILEGES;")
|
||||
raise
|
||||
|
||||
# 初始化表结构
|
||||
async with self.get_connection() as conn:
|
||||
async with conn.cursor() as cursor:
|
||||
await cursor.execute('''
|
||||
CREATE TABLE IF NOT EXISTS email_accounts (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
email VARCHAR(255) UNIQUE NOT NULL,
|
||||
password VARCHAR(255) NOT NULL,
|
||||
client_id VARCHAR(255) NOT NULL,
|
||||
refresh_token TEXT NOT NULL,
|
||||
in_use BOOLEAN DEFAULT 0,
|
||||
cursor_password VARCHAR(255),
|
||||
cursor_cookie TEXT,
|
||||
cursor_token TEXT,
|
||||
sold BOOLEAN DEFAULT 0,
|
||||
status VARCHAR(20) DEFAULT 'pending',
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
INDEX idx_status_inuse_sold (status, in_use, sold)
|
||||
)
|
||||
''')
|
||||
|
||||
# 初始化Redis连接(如果配置了)
|
||||
if self.use_redis and REDIS_AVAILABLE and self.redis_config:
|
||||
try:
|
||||
# 根据检测到的Redis库类型创建连接
|
||||
if REDIS_TYPE == "redis-py":
|
||||
# 使用redis.asyncio创建连接
|
||||
logger.info(f"使用redis-py连接Redis: {self.redis_config.host}:{self.redis_config.port}")
|
||||
self.redis = redis_asyncio.Redis(
|
||||
host=self.redis_config.host,
|
||||
port=self.redis_config.port,
|
||||
password=self.redis_config.password or None,
|
||||
db=self.redis_config.db,
|
||||
decode_responses=True
|
||||
)
|
||||
# 测试连接
|
||||
await self.redis.ping()
|
||||
elif REDIS_TYPE == "aioredis":
|
||||
# 使用旧版aioredis创建连接
|
||||
logger.info(f"使用aioredis连接Redis: {self.redis_config.host}:{self.redis_config.port}")
|
||||
self.redis = await aioredis.from_url(
|
||||
f"redis://{self.redis_config.host}:{self.redis_config.port}",
|
||||
password=self.redis_config.password or None,
|
||||
db=self.redis_config.db,
|
||||
encoding="utf-8",
|
||||
decode_responses=True
|
||||
)
|
||||
logger.info("Redis连接初始化成功")
|
||||
except Exception as e:
|
||||
logger.error(f"Redis连接初始化失败: {e}")
|
||||
logger.info("Redis缓存将被禁用")
|
||||
self.redis = None
|
||||
|
||||
logger.info(f"数据库连接池初始化完成,大小: {self._pool_size}")
|
||||
|
||||
async def cleanup(self):
|
||||
"""清理数据库连接"""
|
||||
for conn in self._pool:
|
||||
await conn.close()
|
||||
self._pool.clear()
|
||||
if self._pool:
|
||||
self._pool.close()
|
||||
await self._pool.wait_closed()
|
||||
|
||||
if self.redis:
|
||||
if REDIS_TYPE == "redis-py":
|
||||
await self.redis.close()
|
||||
else:
|
||||
await self.redis.close()
|
||||
|
||||
logger.info("数据库连接已清理")
|
||||
|
||||
@asynccontextmanager
|
||||
async def get_connection(self):
|
||||
"""获取数据库连接"""
|
||||
async with self._pool_lock:
|
||||
if not self._pool:
|
||||
conn = await aiosqlite.connect(self.db_path)
|
||||
else:
|
||||
conn = self._pool.pop()
|
||||
|
||||
try:
|
||||
yield conn
|
||||
finally:
|
||||
if len(self._pool) < self._pool_size:
|
||||
self._pool.append(conn)
|
||||
else:
|
||||
await conn.close()
|
||||
if self._pool is None:
|
||||
raise Exception("数据库连接池未初始化")
|
||||
|
||||
async with self._pool.acquire() as conn:
|
||||
try:
|
||||
yield conn
|
||||
finally:
|
||||
pass # 连接会自动返回池中
|
||||
|
||||
async def execute(self, query: str, params: tuple = ()) -> Any:
|
||||
"""执行SQL语句"""
|
||||
async with self.get_connection() as conn:
|
||||
cursor = await conn.execute(query, params)
|
||||
await conn.commit()
|
||||
return cursor.lastrowid
|
||||
logger.debug(f"执行SQL: {query}, 参数: {params}")
|
||||
try:
|
||||
async with self.get_connection() as conn:
|
||||
async with conn.cursor() as cursor:
|
||||
await cursor.execute(query, params)
|
||||
|
||||
# 对于INSERT语句,返回最后插入的ID
|
||||
if query.strip().upper().startswith("INSERT"):
|
||||
return cursor.lastrowid
|
||||
|
||||
# 对于UPDATE/DELETE语句,返回影响的行数
|
||||
return cursor.rowcount
|
||||
except Exception as e:
|
||||
logger.error(f"SQL执行失败: {query}, 参数: {params}, 错误: {str(e)}")
|
||||
raise
|
||||
|
||||
async def fetch_one(self, query: str, params: tuple = ()) -> Optional[tuple]:
|
||||
async def fetch_one(self, query: str, params: tuple = ()) -> Optional[Dict]:
|
||||
"""查询单条记录"""
|
||||
async with self.get_connection() as conn:
|
||||
cursor = await conn.execute(query, params)
|
||||
return await cursor.fetchone()
|
||||
logger.debug(f"查询单条: {query}, 参数: {params}")
|
||||
|
||||
# 尝试从Redis获取缓存
|
||||
cache_key = f"db:{self._make_cache_key(query, params)}"
|
||||
cached_result = await self._get_from_cache(cache_key)
|
||||
if cached_result is not None:
|
||||
return cached_result
|
||||
|
||||
try:
|
||||
async with self.get_connection() as conn:
|
||||
async with conn.cursor(aiomysql.DictCursor) as cursor:
|
||||
await cursor.execute(query, params)
|
||||
result = await cursor.fetchone()
|
||||
|
||||
# 缓存结果
|
||||
if result and self.redis:
|
||||
await self._store_in_cache(cache_key, result)
|
||||
|
||||
logger.debug(f"查询结果: {result}")
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.error(f"查询单条失败: {query}, 参数: {params}, 错误: {str(e)}")
|
||||
raise
|
||||
|
||||
async def fetch_all(self, query: str, params: tuple = ()) -> List[tuple]:
|
||||
async def fetch_all(self, query: str, params: tuple = ()) -> List[Dict]:
|
||||
"""查询多条记录"""
|
||||
async with self.get_connection() as conn:
|
||||
cursor = await conn.execute(query, params)
|
||||
return await cursor.fetchall()
|
||||
logger.debug(f"查询多条: {query}, 参数: {params}")
|
||||
|
||||
# 尝试从Redis获取缓存
|
||||
cache_key = f"db:{self._make_cache_key(query, params)}"
|
||||
cached_result = await self._get_from_cache(cache_key)
|
||||
if cached_result is not None:
|
||||
return cached_result
|
||||
|
||||
try:
|
||||
async with self.get_connection() as conn:
|
||||
async with conn.cursor(aiomysql.DictCursor) as cursor:
|
||||
await cursor.execute(query, params)
|
||||
results = await cursor.fetchall()
|
||||
|
||||
# 缓存结果
|
||||
if results and self.redis:
|
||||
await self._store_in_cache(cache_key, results)
|
||||
|
||||
logger.debug(f"查询结果数量: {len(results)}")
|
||||
return results
|
||||
except Exception as e:
|
||||
logger.error(f"查询多条失败: {query}, 参数: {params}, 错误: {str(e)}")
|
||||
raise
|
||||
|
||||
async def _get_from_cache(self, key: str) -> Optional[Union[Dict, List[Dict]]]:
|
||||
"""从Redis缓存获取数据"""
|
||||
if not self.redis:
|
||||
return None
|
||||
|
||||
try:
|
||||
cached_data = await self.redis.get(key)
|
||||
if cached_data:
|
||||
return json.loads(cached_data)
|
||||
except Exception as e:
|
||||
logger.error(f"从缓存获取数据失败: {e}")
|
||||
|
||||
return None
|
||||
|
||||
async def _store_in_cache(self, key: str, data: Union[Dict, List[Dict]], ttl: int = 300) -> bool:
|
||||
"""存储数据到Redis缓存"""
|
||||
if not self.redis:
|
||||
return False
|
||||
|
||||
try:
|
||||
json_data = json.dumps(data)
|
||||
if REDIS_TYPE == "redis-py":
|
||||
await self.redis.setex(key, ttl, json_data)
|
||||
else:
|
||||
await self.redis.setex(key, ttl, json_data)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"存储数据到缓存失败: {e}")
|
||||
return False
|
||||
|
||||
async def clear_cache(self, pattern: str = "db:*") -> int:
|
||||
"""清除缓存"""
|
||||
if not self.redis:
|
||||
return 0
|
||||
|
||||
try:
|
||||
if REDIS_TYPE == "redis-py":
|
||||
keys = await self.redis.keys(pattern)
|
||||
if not keys:
|
||||
return 0
|
||||
return await self.redis.delete(*keys)
|
||||
else:
|
||||
keys = await self.redis.keys(pattern)
|
||||
if not keys:
|
||||
return 0
|
||||
return await self.redis.delete(*keys)
|
||||
except Exception as e:
|
||||
logger.error(f"清除缓存失败: {e}")
|
||||
return 0
|
||||
|
||||
def _make_cache_key(self, query: str, params: tuple) -> str:
|
||||
"""生成缓存键"""
|
||||
return f"{query}:{hash(params)}"
|
||||
1
database_schema.sql
Normal file
1
database_schema.sql
Normal file
@@ -0,0 +1 @@
|
||||
|
||||
137
import_emails.py
137
import_emails.py
@@ -1,61 +1,110 @@
|
||||
import asyncio
|
||||
import sys
|
||||
|
||||
# Windows平台特殊处理,强制使用SelectorEventLoop
|
||||
if sys.platform.startswith('win'):
|
||||
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
|
||||
|
||||
import aiosqlite
|
||||
from loguru import logger
|
||||
|
||||
from core.config import Config
|
||||
from core.database import DatabaseManager
|
||||
|
||||
|
||||
async def import_emails(config: Config, file_path: str):
|
||||
"""导入邮箱账号到数据库"""
|
||||
async def import_emails(config: Config, db_manager: DatabaseManager, file_path: str):
|
||||
"""导入邮箱账号到MySQL数据库"""
|
||||
DEFAULT_CLIENT_ID = "9e5f94bc-e8a4-4e73-b8be-63364c29d753"
|
||||
|
||||
async with aiosqlite.connect(config.database_config.path) as db:
|
||||
# 创建表
|
||||
await db.execute('''
|
||||
CREATE TABLE IF NOT EXISTS email_accounts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
email TEXT UNIQUE NOT NULL,
|
||||
password TEXT NOT NULL,
|
||||
client_id TEXT NOT NULL,
|
||||
refresh_token TEXT NOT NULL,
|
||||
in_use BOOLEAN DEFAULT 0,
|
||||
cursor_password TEXT,
|
||||
cursor_cookie TEXT,
|
||||
cursor_token TEXT,
|
||||
sold BOOLEAN DEFAULT 0,
|
||||
status TEXT DEFAULT 'pending',
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
)
|
||||
''')
|
||||
|
||||
# 读取文件并导入数据
|
||||
count = 0
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
if line.strip():
|
||||
try:
|
||||
email, password, client_id, refresh_token = line.strip().split('----')
|
||||
await db.execute('''
|
||||
INSERT INTO email_accounts (
|
||||
email, password, client_id, refresh_token, status
|
||||
) VALUES (?, ?, ?, ?, 'pending')
|
||||
''', (email, password, client_id, refresh_token))
|
||||
count += 1
|
||||
except aiosqlite.IntegrityError:
|
||||
logger.warning(f"重复的邮箱: {email}")
|
||||
except ValueError:
|
||||
logger.error(f"无效的数据行: {line.strip()}")
|
||||
# 确保数据库连接已初始化
|
||||
if not db_manager._pool:
|
||||
await db_manager.initialize()
|
||||
|
||||
# 读取文件并导入数据
|
||||
count = 0
|
||||
duplicate_count = 0
|
||||
error_count = 0
|
||||
|
||||
logger.info(f"开始从 {file_path} 导入邮箱账号")
|
||||
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
for line_num, line in enumerate(f, 1):
|
||||
if not line.strip():
|
||||
continue
|
||||
|
||||
try:
|
||||
# 解析数据行
|
||||
parts = line.strip().split('----')
|
||||
if len(parts) < 4:
|
||||
logger.error(f"行 {line_num}: 格式不正确,期望 'email----password----client_id----refresh_token'")
|
||||
error_count += 1
|
||||
continue
|
||||
|
||||
email, password, client_id, refresh_token = parts
|
||||
|
||||
# 插入数据库
|
||||
insert_query = '''
|
||||
INSERT INTO email_accounts
|
||||
(email, password, client_id, refresh_token, status)
|
||||
VALUES (%s, %s, %s, %s, 'pending')
|
||||
'''
|
||||
|
||||
try:
|
||||
await db_manager.execute(insert_query, (email, password, client_id, refresh_token))
|
||||
count += 1
|
||||
|
||||
if count % 100 == 0:
|
||||
logger.info(f"已导入 {count} 个邮箱账号")
|
||||
|
||||
await db.commit()
|
||||
logger.success(f"成功导入 {count} 个邮箱账号")
|
||||
except Exception as e:
|
||||
if "Duplicate entry" in str(e):
|
||||
logger.warning(f"行 {line_num}: 重复的邮箱: {email}")
|
||||
duplicate_count += 1
|
||||
else:
|
||||
logger.error(f"行 {line_num}: 导入失败: {str(e)}")
|
||||
error_count += 1
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"行 {line_num}: 处理时出错: {str(e)}")
|
||||
error_count += 1
|
||||
|
||||
# 如果启用了Redis缓存,清除相关缓存
|
||||
if db_manager.redis:
|
||||
cleared = await db_manager.clear_cache("db:*")
|
||||
logger.info(f"已清除 {cleared} 个Redis缓存键")
|
||||
|
||||
logger.success(f"导入完成: 成功 {count} 个, 重复 {duplicate_count} 个, 失败 {error_count} 个")
|
||||
return count
|
||||
|
||||
|
||||
async def main():
|
||||
config = Config.from_yaml()
|
||||
await import_emails(config, "email.txt")
|
||||
try:
|
||||
# 加载配置
|
||||
config = Config.from_yaml()
|
||||
|
||||
# 初始化数据库管理器
|
||||
db_manager = DatabaseManager(config)
|
||||
await db_manager.initialize()
|
||||
|
||||
# 从配置中获取邮箱文件路径,或使用默认值
|
||||
file_path = config.email_config.file_path if hasattr(config, 'email_config') and config.email_config else "email.txt"
|
||||
|
||||
# 导入邮箱
|
||||
await import_emails(config, db_manager, file_path)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"程序执行出错: {str(e)}")
|
||||
finally:
|
||||
# 清理资源
|
||||
if 'db_manager' in locals():
|
||||
await db_manager.cleanup()
|
||||
logger.info("程序执行完毕")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# 设置日志
|
||||
logger.remove()
|
||||
logger.add(sys.stderr, level="INFO")
|
||||
logger.add("import_emails.log", rotation="1 MB", level="DEBUG")
|
||||
|
||||
# 执行导入
|
||||
asyncio.run(main())
|
||||
327
init_database.py
Normal file
327
init_database.py
Normal file
@@ -0,0 +1,327 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
数据库初始化脚本
|
||||
用于自动创建MySQL数据库、用户和表结构,并配置Redis
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import getpass
|
||||
import asyncio
|
||||
import yaml
|
||||
import pymysql
|
||||
import redis
|
||||
|
||||
from loguru import logger
|
||||
|
||||
|
||||
# 默认配置
|
||||
DEFAULT_CONFIG = {
|
||||
"database": {
|
||||
"host": "localhost",
|
||||
"port": 3306,
|
||||
"username": "auto_cursor_reg",
|
||||
"password": "auto_cursor_pass",
|
||||
"database": "auto_cursor_reg",
|
||||
"pool_size": 10,
|
||||
"use_redis": True
|
||||
},
|
||||
"redis": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 6379,
|
||||
"password": "",
|
||||
"db": 0
|
||||
}
|
||||
}
|
||||
|
||||
# 数据库表结构
|
||||
EMAIL_ACCOUNTS_TABLE = '''
|
||||
CREATE TABLE IF NOT EXISTS email_accounts (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
email VARCHAR(255) UNIQUE NOT NULL,
|
||||
password VARCHAR(255) NOT NULL,
|
||||
client_id VARCHAR(255) NOT NULL,
|
||||
refresh_token TEXT NOT NULL,
|
||||
in_use BOOLEAN DEFAULT 0,
|
||||
cursor_password VARCHAR(255),
|
||||
cursor_cookie TEXT,
|
||||
cursor_token TEXT,
|
||||
sold BOOLEAN DEFAULT 0,
|
||||
status VARCHAR(20) DEFAULT 'pending',
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
INDEX idx_status_inuse_sold (status, in_use, sold)
|
||||
)
|
||||
'''
|
||||
|
||||
|
||||
def setup_logger():
|
||||
"""设置日志"""
|
||||
logger.remove()
|
||||
logger.add(sys.stderr, level="INFO")
|
||||
logger.add("init_database.log", rotation="1 MB", level="DEBUG")
|
||||
|
||||
|
||||
def get_mysql_root_credentials():
|
||||
"""获取MySQL root用户凭据"""
|
||||
print("\n===== MySQL配置 =====")
|
||||
print("请输入MySQL root用户凭据以创建数据库和用户")
|
||||
|
||||
mysql_host = input("MySQL 主机地址 [localhost]: ") or "localhost"
|
||||
mysql_port = input("MySQL 端口 [3306]: ") or "3306"
|
||||
try:
|
||||
mysql_port = int(mysql_port)
|
||||
except ValueError:
|
||||
mysql_port = 3306
|
||||
print("端口无效,使用默认端口 3306")
|
||||
|
||||
mysql_root = input("MySQL root用户名 [root]: ") or "root"
|
||||
mysql_root_password = getpass.getpass("MySQL root密码: ")
|
||||
|
||||
return {
|
||||
"host": mysql_host,
|
||||
"port": mysql_port,
|
||||
"user": mysql_root,
|
||||
"password": mysql_root_password
|
||||
}
|
||||
|
||||
|
||||
def get_app_database_config():
|
||||
"""获取应用数据库配置"""
|
||||
print("\n请设置应用程序的数据库配置:")
|
||||
|
||||
db_name = input("数据库名称 [auto_cursor_reg]: ") or "auto_cursor_reg"
|
||||
db_user = input("数据库用户名 [auto_cursor_reg]: ") or "auto_cursor_reg"
|
||||
db_pass = getpass.getpass(f"为用户 {db_user} 设置密码 [auto_cursor_pass]: ")
|
||||
if not db_pass:
|
||||
db_pass = "auto_cursor_pass"
|
||||
|
||||
return {
|
||||
"database": db_name,
|
||||
"username": db_user,
|
||||
"password": db_pass
|
||||
}
|
||||
|
||||
|
||||
def get_redis_config():
|
||||
"""获取Redis配置"""
|
||||
use_redis = input("\n是否使用Redis缓存? (y/n) [y]: ").lower() != "n"
|
||||
|
||||
if not use_redis:
|
||||
return {"use_redis": False}
|
||||
|
||||
print("\n===== Redis配置 =====")
|
||||
redis_host = input("Redis 主机地址 [127.0.0.1]: ") or "127.0.0.1"
|
||||
redis_port = input("Redis 端口 [6379]: ") or "6379"
|
||||
try:
|
||||
redis_port = int(redis_port)
|
||||
except ValueError:
|
||||
redis_port = 6379
|
||||
print("端口无效,使用默认端口 6379")
|
||||
|
||||
redis_password = getpass.getpass("Redis 密码 (如无密码请直接回车): ")
|
||||
redis_db = input("Redis 数据库索引 [0]: ") or "0"
|
||||
try:
|
||||
redis_db = int(redis_db)
|
||||
except ValueError:
|
||||
redis_db = 0
|
||||
print("数据库索引无效,使用默认值 0")
|
||||
|
||||
return {
|
||||
"use_redis": True,
|
||||
"redis": {
|
||||
"host": redis_host,
|
||||
"port": redis_port,
|
||||
"password": redis_password,
|
||||
"db": redis_db
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def test_mysql_connection(config):
|
||||
"""测试MySQL连接"""
|
||||
try:
|
||||
conn = pymysql.connect(**config)
|
||||
conn.close()
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"MySQL连接失败: {str(e)}")
|
||||
return False
|
||||
|
||||
|
||||
def test_redis_connection(config):
|
||||
"""测试Redis连接"""
|
||||
try:
|
||||
r = redis.Redis(
|
||||
host=config["host"],
|
||||
port=config["port"],
|
||||
password=config["password"] if config["password"] else None,
|
||||
db=config["db"]
|
||||
)
|
||||
r.ping()
|
||||
r.close()
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"Redis连接失败: {str(e)}")
|
||||
return False
|
||||
|
||||
|
||||
def setup_mysql(root_config, app_config):
|
||||
"""设置MySQL数据库和用户"""
|
||||
logger.info("开始设置MySQL数据库...")
|
||||
|
||||
try:
|
||||
# 连接到MySQL
|
||||
conn = pymysql.connect(**root_config)
|
||||
cursor = conn.cursor()
|
||||
|
||||
db_name = app_config["database"]
|
||||
db_user = app_config["username"]
|
||||
db_pass = app_config["password"]
|
||||
|
||||
# 创建数据库
|
||||
logger.info(f"创建数据库: {db_name}")
|
||||
cursor.execute(f"CREATE DATABASE IF NOT EXISTS `{db_name}` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci")
|
||||
|
||||
# 创建用户并授权
|
||||
logger.info(f"创建用户: {db_user}")
|
||||
|
||||
# 检查用户是否已存在
|
||||
cursor.execute(f"SELECT User FROM mysql.user WHERE User = '{db_user}'")
|
||||
user_exists = cursor.fetchone()
|
||||
|
||||
if user_exists:
|
||||
logger.info(f"用户 {db_user} 已存在,更新密码")
|
||||
cursor.execute(f"ALTER USER '{db_user}'@'localhost' IDENTIFIED BY '{db_pass}'")
|
||||
cursor.execute(f"ALTER USER '{db_user}'@'%' IDENTIFIED BY '{db_pass}'")
|
||||
else:
|
||||
# 创建用户 (同时创建本地和远程连接权限)
|
||||
try:
|
||||
cursor.execute(f"CREATE USER '{db_user}'@'localhost' IDENTIFIED BY '{db_pass}'")
|
||||
cursor.execute(f"CREATE USER '{db_user}'@'%' IDENTIFIED BY '{db_pass}'")
|
||||
except pymysql.err.MySQLError as e:
|
||||
logger.warning(f"创建用户时出现警告 (可能用户已存在): {str(e)}")
|
||||
|
||||
# 授权
|
||||
cursor.execute(f"GRANT ALL PRIVILEGES ON `{db_name}`.* TO '{db_user}'@'localhost'")
|
||||
cursor.execute(f"GRANT ALL PRIVILEGES ON `{db_name}`.* TO '{db_user}'@'%'")
|
||||
cursor.execute("FLUSH PRIVILEGES")
|
||||
|
||||
# 切换到新创建的数据库
|
||||
cursor.execute(f"USE `{db_name}`")
|
||||
|
||||
# 创建表
|
||||
logger.info("创建数据表: email_accounts")
|
||||
cursor.execute(EMAIL_ACCOUNTS_TABLE)
|
||||
|
||||
conn.commit()
|
||||
cursor.close()
|
||||
conn.close()
|
||||
|
||||
logger.success("MySQL数据库设置成功")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"设置MySQL失败: {str(e)}")
|
||||
return False
|
||||
|
||||
|
||||
def update_config_file(db_config, redis_config=None):
|
||||
"""更新配置文件"""
|
||||
logger.info("更新配置文件...")
|
||||
|
||||
config_file = "config.yaml"
|
||||
|
||||
try:
|
||||
# 读取现有配置
|
||||
with open(config_file, "r", encoding="utf-8") as f:
|
||||
config = yaml.safe_load(f)
|
||||
|
||||
# 更新数据库配置
|
||||
config["database"].update({
|
||||
"host": db_config.get("host", "localhost"),
|
||||
"port": db_config.get("port", 3306),
|
||||
"username": db_config["username"],
|
||||
"password": db_config["password"],
|
||||
"database": db_config["database"],
|
||||
"use_redis": db_config.get("use_redis", False)
|
||||
})
|
||||
|
||||
# 如果启用Redis,更新Redis配置
|
||||
if redis_config and db_config.get("use_redis"):
|
||||
config["redis"] = redis_config
|
||||
|
||||
# 备份原配置文件
|
||||
if os.path.exists(config_file):
|
||||
os.rename(config_file, f"{config_file}.bak")
|
||||
logger.info(f"已备份原配置文件为 {config_file}.bak")
|
||||
|
||||
# 写入新配置
|
||||
with open(config_file, "w", encoding="utf-8") as f:
|
||||
yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
|
||||
|
||||
logger.success(f"配置文件已更新: {config_file}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"更新配置文件失败: {str(e)}")
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""主函数"""
|
||||
setup_logger()
|
||||
logger.info("开始初始化数据库")
|
||||
|
||||
# 获取MySQL root凭据
|
||||
root_config = get_mysql_root_credentials()
|
||||
|
||||
# 测试MySQL连接
|
||||
if not test_mysql_connection(root_config):
|
||||
logger.error("无法连接到MySQL,请检查凭据和服务状态")
|
||||
return
|
||||
|
||||
# 获取应用数据库配置
|
||||
db_config = get_app_database_config()
|
||||
|
||||
# 获取Redis配置
|
||||
redis_info = get_redis_config()
|
||||
db_config["use_redis"] = redis_info["use_redis"]
|
||||
|
||||
# 如果启用了Redis,测试连接
|
||||
if redis_info["use_redis"]:
|
||||
redis_config = redis_info["redis"]
|
||||
if not test_redis_connection(redis_config):
|
||||
use_anyway = input("Redis连接测试失败,是否继续? (y/n) [n]: ").lower() == "y"
|
||||
if not use_anyway:
|
||||
logger.warning("用户取消了初始化")
|
||||
return
|
||||
else:
|
||||
redis_config = None
|
||||
|
||||
# 设置MySQL
|
||||
if not setup_mysql(root_config, db_config):
|
||||
return
|
||||
|
||||
# 合并配置
|
||||
final_db_config = {
|
||||
"host": root_config["host"],
|
||||
"port": root_config["port"],
|
||||
"username": db_config["username"],
|
||||
"password": db_config["password"],
|
||||
"database": db_config["database"],
|
||||
"use_redis": db_config["use_redis"]
|
||||
}
|
||||
|
||||
# 更新配置文件
|
||||
update_config_file(final_db_config, redis_config)
|
||||
|
||||
logger.success("数据库初始化完成!")
|
||||
print("\n===== 初始化完成 =====")
|
||||
print("您现在可以运行以下命令导入邮箱账号:")
|
||||
print(" python import_emails.py")
|
||||
print("\n然后运行主程序开始注册:")
|
||||
print(" python main.py")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
177
migrate_db.py
Normal file
177
migrate_db.py
Normal file
@@ -0,0 +1,177 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
SQLite 到 MySQL 数据迁移脚本
|
||||
用于将旧的 SQLite 数据库迁移到新的 MySQL 数据库
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
import sys
|
||||
from typing import List, Dict, Any
|
||||
|
||||
import aiomysql
|
||||
import yaml
|
||||
from loguru import logger
|
||||
|
||||
|
||||
async def migrate_data():
|
||||
"""迁移数据主函数"""
|
||||
# 读取配置文件
|
||||
logger.info("读取配置文件")
|
||||
with open("config.yaml", "r", encoding="utf-8") as f:
|
||||
config = yaml.safe_load(f)
|
||||
|
||||
# 获取数据库配置
|
||||
db_config = config.get("database", {})
|
||||
sqlite_path = db_config.get("path", "cursor.db")
|
||||
mysql_config = {
|
||||
"host": db_config.get("host", "localhost"),
|
||||
"port": db_config.get("port", 3306),
|
||||
"user": db_config.get("username", "auto_cursor_reg"), # 使用正确的默认用户名
|
||||
"password": db_config.get("password", "this_password_jiaqiao"), # 使用正确的默认密码
|
||||
"db": db_config.get("database", "auto_cursor_reg"), # 使用正确的默认数据库名
|
||||
"charset": "utf8mb4"
|
||||
}
|
||||
|
||||
logger.info(f"MySQL配置: 主机={mysql_config['host']}:{mysql_config['port']}, 用户={mysql_config['user']}, 数据库={mysql_config['db']}")
|
||||
|
||||
# 连接SQLite数据库
|
||||
logger.info(f"连接SQLite数据库: {sqlite_path}")
|
||||
sqlite_conn = None
|
||||
mysql_conn = None
|
||||
|
||||
try:
|
||||
sqlite_conn = sqlite3.connect(sqlite_path)
|
||||
sqlite_conn.row_factory = sqlite3.Row # 启用字典行工厂
|
||||
except Exception as e:
|
||||
logger.error(f"无法连接SQLite数据库: {e}")
|
||||
return
|
||||
|
||||
# 连接MySQL数据库
|
||||
logger.info(f"尝试连接MySQL数据库...")
|
||||
try:
|
||||
mysql_conn = await aiomysql.connect(**mysql_config)
|
||||
logger.info("MySQL数据库连接成功")
|
||||
except Exception as e:
|
||||
logger.error(f"无法连接MySQL数据库: {e}")
|
||||
logger.error(f"请确认MySQL服务已启动且用户名密码正确")
|
||||
logger.info(f"您可能需要创建MySQL用户和数据库:")
|
||||
logger.info(f" CREATE USER '{mysql_config['user']}'@'localhost' IDENTIFIED BY '{mysql_config['password']}';")
|
||||
logger.info(f" CREATE DATABASE {mysql_config['db']} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;")
|
||||
logger.info(f" GRANT ALL PRIVILEGES ON {mysql_config['db']}.* TO '{mysql_config['user']}'@'localhost';")
|
||||
logger.info(f" FLUSH PRIVILEGES;")
|
||||
if sqlite_conn:
|
||||
sqlite_conn.close()
|
||||
return
|
||||
|
||||
try:
|
||||
# 检查email_accounts表是否存在
|
||||
logger.info("检查并创建MySQL表结构")
|
||||
async with mysql_conn.cursor() as cursor:
|
||||
await cursor.execute('''
|
||||
CREATE TABLE IF NOT EXISTS email_accounts (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
email VARCHAR(255) UNIQUE NOT NULL,
|
||||
password VARCHAR(255) NOT NULL,
|
||||
client_id VARCHAR(255) NOT NULL,
|
||||
refresh_token TEXT NOT NULL,
|
||||
in_use BOOLEAN DEFAULT 0,
|
||||
cursor_password VARCHAR(255),
|
||||
cursor_cookie TEXT,
|
||||
cursor_token TEXT,
|
||||
sold BOOLEAN DEFAULT 0,
|
||||
status VARCHAR(20) DEFAULT 'pending',
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
INDEX idx_status_inuse_sold (status, in_use, sold)
|
||||
)
|
||||
''')
|
||||
await mysql_conn.commit()
|
||||
|
||||
# 从SQLite读取数据
|
||||
logger.info("从SQLite读取数据")
|
||||
try:
|
||||
sqlite_cursor = sqlite_conn.cursor()
|
||||
sqlite_cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='email_accounts'")
|
||||
if not sqlite_cursor.fetchone():
|
||||
logger.warning("SQLite数据库中不存在email_accounts表,无需迁移")
|
||||
return
|
||||
|
||||
# 获取表结构信息
|
||||
sqlite_cursor.execute("PRAGMA table_info(email_accounts)")
|
||||
columns_info = sqlite_cursor.fetchall()
|
||||
column_names = [col["name"] for col in columns_info]
|
||||
|
||||
# 读取所有数据
|
||||
sqlite_cursor.execute("SELECT * FROM email_accounts")
|
||||
rows = sqlite_cursor.fetchall()
|
||||
logger.info(f"从SQLite读取到 {len(rows)} 条数据")
|
||||
except Exception as e:
|
||||
logger.error(f"读取SQLite数据失败: {e}")
|
||||
return
|
||||
|
||||
# 迁移数据到MySQL
|
||||
if rows:
|
||||
logger.info("开始迁移数据到MySQL")
|
||||
try:
|
||||
async with mysql_conn.cursor() as cursor:
|
||||
# 检查MySQL表中是否已有数据
|
||||
await cursor.execute("SELECT COUNT(*) FROM email_accounts")
|
||||
result = await cursor.fetchone()
|
||||
if result and result[0] > 0:
|
||||
logger.warning("MySQL表中已存在数据,是否继续?(y/n)")
|
||||
response = input().strip().lower()
|
||||
if response != 'y':
|
||||
logger.info("用户取消迁移")
|
||||
return
|
||||
|
||||
# 构建插入查询
|
||||
placeholders = ", ".join(["%s"] * len(column_names))
|
||||
columns = ", ".join(column_names)
|
||||
query = f"INSERT INTO email_accounts ({columns}) VALUES ({placeholders}) ON DUPLICATE KEY UPDATE id=id"
|
||||
|
||||
# 批量插入数据
|
||||
batch_size = 100
|
||||
for i in range(0, len(rows), batch_size):
|
||||
batch = rows[i:i+batch_size]
|
||||
batch_values = []
|
||||
for row in batch:
|
||||
# 将行数据转换为列表
|
||||
row_values = [row[name] for name in column_names]
|
||||
batch_values.append(row_values)
|
||||
|
||||
await cursor.executemany(query, batch_values)
|
||||
await mysql_conn.commit()
|
||||
logger.info(f"已迁移 {min(i+batch_size, len(rows))}/{len(rows)} 条数据")
|
||||
|
||||
logger.success(f"数据迁移完成,共迁移 {len(rows)} 条数据")
|
||||
except Exception as e:
|
||||
logger.error(f"迁移数据到MySQL失败: {e}")
|
||||
else:
|
||||
logger.warning("SQLite数据库中没有数据需要迁移")
|
||||
|
||||
finally:
|
||||
# 安全关闭连接
|
||||
if sqlite_conn:
|
||||
sqlite_conn.close()
|
||||
logger.debug("SQLite连接已关闭")
|
||||
|
||||
if mysql_conn:
|
||||
await mysql_conn.close()
|
||||
logger.debug("MySQL连接已关闭")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Windows平台特殊处理,强制使用SelectorEventLoop
|
||||
if sys.platform.startswith('win'):
|
||||
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
|
||||
|
||||
# 设置日志
|
||||
logger.remove()
|
||||
logger.add(sys.stderr, level="INFO")
|
||||
logger.add("migrate.log", rotation="1 MB", level="DEBUG")
|
||||
|
||||
# 执行迁移
|
||||
logger.info("开始执行数据迁移")
|
||||
asyncio.run(migrate_data())
|
||||
logger.info("数据迁移脚本执行完毕")
|
||||
@@ -22,6 +22,13 @@ python-dateutil
|
||||
|
||||
# Database
|
||||
aiosqlite
|
||||
aiomysql>=0.1.1
|
||||
pymysql>=1.0.2
|
||||
|
||||
# Redis - 使用其中一个
|
||||
# 如果使用Python 3.12,推荐使用redis-py而不是aioredis
|
||||
redis>=4.2.0
|
||||
# aioredis>=2.0.0 # 不建议在Python 3.12上使用
|
||||
|
||||
# Logging
|
||||
loguru==0.7.2
|
||||
|
||||
@@ -39,29 +39,44 @@ class EmailManager:
|
||||
"""批量获取未使用的邮箱账号"""
|
||||
logger.info(f"尝试获取 {num} 个未使用的邮箱账号")
|
||||
|
||||
query = '''
|
||||
UPDATE email_accounts
|
||||
SET in_use = 1, updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id IN (
|
||||
SELECT id FROM email_accounts
|
||||
WHERE in_use = 0 AND sold = 0 AND status = 'pending'
|
||||
LIMIT ?
|
||||
)
|
||||
RETURNING id, email, password, client_id, refresh_token
|
||||
'''
|
||||
# 1. 先查询符合条件的账号ID列表
|
||||
select_query = """
|
||||
SELECT id, email, password, client_id, refresh_token
|
||||
FROM email_accounts
|
||||
WHERE in_use = 0 AND sold = 0 AND status = 'pending'
|
||||
LIMIT %s
|
||||
"""
|
||||
accounts = await self.db.fetch_all(select_query, (num,))
|
||||
|
||||
if not accounts:
|
||||
logger.debug("没有找到符合条件的账号")
|
||||
return []
|
||||
|
||||
results = await self.db.fetch_all(query, (num,))
|
||||
logger.debug(f"实际获取到 {len(results)} 个账号")
|
||||
# 2. 提取账号ID列表
|
||||
account_ids = [account['id'] for account in accounts]
|
||||
|
||||
# 3. 更新这些账号的状态
|
||||
if account_ids:
|
||||
placeholders = ', '.join(['%s' for _ in account_ids])
|
||||
update_query = f"""
|
||||
UPDATE email_accounts
|
||||
SET in_use = 1, updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id IN ({placeholders})
|
||||
"""
|
||||
await self.db.execute(update_query, tuple(account_ids))
|
||||
|
||||
# 4. 返回账号数据
|
||||
logger.debug(f"实际获取到 {len(accounts)} 个账号")
|
||||
return [
|
||||
EmailAccount(
|
||||
id=row[0],
|
||||
email=row[1],
|
||||
password=row[2],
|
||||
client_id=row[3],
|
||||
refresh_token=row[4],
|
||||
id=row['id'],
|
||||
email=row['email'],
|
||||
password=row['password'],
|
||||
client_id=row['client_id'],
|
||||
refresh_token=row['refresh_token'],
|
||||
in_use=True
|
||||
)
|
||||
for row in results
|
||||
for row in accounts
|
||||
]
|
||||
|
||||
async def update_account_status(self, account_id: int, status: str):
|
||||
@@ -69,10 +84,10 @@ class EmailManager:
|
||||
query = '''
|
||||
UPDATE email_accounts
|
||||
SET
|
||||
status = ?,
|
||||
status = %s,
|
||||
in_use = 0,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = ?
|
||||
WHERE id = %s
|
||||
'''
|
||||
await self.db.execute(query, (status, account_id))
|
||||
|
||||
@@ -81,14 +96,14 @@ class EmailManager:
|
||||
query = '''
|
||||
UPDATE email_accounts
|
||||
SET
|
||||
cursor_password = ?,
|
||||
cursor_cookie = ?,
|
||||
cursor_token = ?,
|
||||
cursor_password = %s,
|
||||
cursor_cookie = %s,
|
||||
cursor_token = %s,
|
||||
in_use = 0,
|
||||
sold = 1,
|
||||
status = 'success',
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = ?
|
||||
WHERE id = %s
|
||||
'''
|
||||
await self.db.execute(query, (cursor_password, cursor_cookie, cursor_token, account_id))
|
||||
|
||||
@@ -97,7 +112,7 @@ class EmailManager:
|
||||
query = '''
|
||||
UPDATE email_accounts
|
||||
SET in_use = 0, updated_at = CURRENT_TIMESTAMP
|
||||
WHERE id = ?
|
||||
WHERE id = %s
|
||||
'''
|
||||
await self.db.execute(query, (account_id,))
|
||||
|
||||
@@ -236,10 +251,10 @@ class EmailManager:
|
||||
logger.debug(f"从文本中提取到验证码: {code}")
|
||||
return code
|
||||
|
||||
logger.warning(f"[{email}] 未能从邮件中提取到验证码")
|
||||
logger.debug(f"[{email}] 邮件内容预览: " + body[:200])
|
||||
logger.warning(f"未能从邮件中提取到验证码")
|
||||
logger.debug(f"邮件内容预览: " + body[:200])
|
||||
return None
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[{email}] 提取验证码失败: {str(e)}")
|
||||
logger.error(f"提取验证码失败: {str(e)}")
|
||||
return None
|
||||
@@ -14,7 +14,7 @@ class ProxyPool:
|
||||
async def batch_get(self, num: int) -> List[str]:
|
||||
"""获取num个代理"""
|
||||
# 临时代理
|
||||
return ['http://127.0.0.1:3057'] * num
|
||||
return ['http://60.188.79.110:20051'] * num
|
||||
|
||||
try:
|
||||
response = await self.fetch_manager.request(
|
||||
|
||||
Reference in New Issue
Block a user