准备创建mysqlv1分支的提交

This commit is contained in:
huangzhenpc
2025-04-01 15:43:27 +08:00
parent cc2a3a34e3
commit 7dad9f6b2f
13 changed files with 1248 additions and 149 deletions

83
IMPORT_README.md Normal file
View File

@@ -0,0 +1,83 @@
# 邮箱导入工具使用说明
这个导入工具用于将邮箱账号导入到MySQL数据库中支持Redis缓存。
## 功能特点
1. 支持MySQL数据库存储
2. 可选启用Redis缓存
3. 详细的导入日志
4. 自动处理重复邮箱
5. Windows平台兼容性优化
## 前置条件
1. MySQL/MariaDB数据库服务已运行
2. 已在`config.yaml`中配置好数据库连接信息
3. Redis服务可选
## 邮箱数据格式
邮箱数据文件应使用以下格式,每行一个账号:
```
email@example.com----密码----client_id----refresh_token
```
字段说明:
- `email`: 邮箱地址
- `password`: 邮箱密码
- `client_id`: Microsoft应用的客户端ID
- `refresh_token`: Microsoft OAuth的刷新令牌
## 使用方法
1. 确保MySQL数据库已正确配置
编辑`config.yaml`文件设置正确的MySQL连接信息
```yaml
database:
host: "localhost"
port: 3306
username: "auto_cursor_reg"
password: "your_password"
database: "auto_cursor_reg"
```
2. 准备邮箱数据文件
默认读取`email.txt`文件,也可以在`config.yaml`中指定:
```yaml
email:
file_path: "path/to/your/email_file.txt"
```
3. 运行导入工具
```bash
python import_emails.py
```
4. 查看导入结果
导入过程和结果会显示在控制台,详细日志保存在`import_emails.log`文件中。
## 常见问题
1. **无法连接数据库**
- 检查MySQL服务是否启动
- 确认用户名和密码正确
- 确认数据库名称存在
2. **导入失败**
- 检查邮箱数据文件格式是否正确
- 查看导入日志获取详细错误信息
3. **重复邮箱处理**
- 系统会自动跳过重复的邮箱,并在日志中标记
## 注意事项
- 导入前建议备份原有数据
- 大批量导入时建议适当增加MySQL的连接超时设置
- 导入成功后可以运行主程序开始注册流程

91
INIT_README.md Normal file
View File

@@ -0,0 +1,91 @@
# 数据库初始化工具使用说明
这个初始化工具用于自动配置MySQL数据库和Redis(可选),创建所需的表结构,并更新配置文件。此工具适用于服务器端首次部署时的快速配置。
## 功能特点
1. 交互式配置MySQL数据库和用户
2. 自动创建所需的表结构
3. 可选配置Redis缓存
4. 自动更新config.yaml配置文件
5. 详细的操作日志
## 前置条件
1. 已安装MySQL/MariaDB服务
2. 已安装Redis服务(可选)
3. 知道MySQL root用户密码
4. 安装必要的Python依赖`pip install -r requirements.txt`
## 使用方法
1. 安装必要的依赖
```bash
pip install -r requirements.txt
```
2. 运行初始化脚本
```bash
python init_database.py
```
3. 根据提示输入信息
- MySQL root用户名和密码
- 应用程序数据库和用户设置
- Redis配置(可选)
4. 初始化完成后,脚本会:
- 创建数据库和用户
- 设置适当的权限
- 创建必要的表结构
- 更新配置文件
## 配置选项说明
### MySQL配置
- **主机地址**: MySQL服务器地址默认为`localhost`
- **端口**: MySQL服务端口默认为`3306`
- **Root用户**: 有权限创建数据库和用户的MySQL管理员账号
- **数据库名**: 应用程序使用的数据库名,默认为`auto_cursor_reg`
- **应用用户名**: 应用程序使用的数据库用户,默认为`auto_cursor_reg`
### Redis配置
- **是否启用**: 是否使用Redis缓存
- **主机地址**: Redis服务器地址默认为`127.0.0.1`
- **端口**: Redis服务端口默认为`6379`
- **密码**: Redis认证密码(如果设置了)
- **数据库索引**: Redis数据库索引默认为`0`
## 常见问题
1. **无法连接到MySQL**
- 确认MySQL服务已启动
- 验证root密码是否正确
- 检查防火墙设置
2. **无法连接到Redis**
- 确认Redis服务已启动
- 验证Redis密码是否正确
- 如不需要Redis可选择禁用
3. **权限问题**
- 确保使用的用户有创建数据库和用户的权限
4. **配置文件备份**
- 脚本会自动备份原始配置文件为`config.yaml.bak`
## 完成后的步骤
初始化完成后,您可以:
1. 导入邮箱账号:
```bash
python import_emails.py
```
2. 运行主程序:
```bash
python main.py
```

107
MYSQL_README.md Normal file
View File

@@ -0,0 +1,107 @@
# Cursor注册工具 - MySQL & Redis支持
本文档说明了如何将Cursor注册工具从SQLite数据库迁移到MySQL数据库并可选地启用Redis缓存以提高性能。
## 变更内容
1. 数据库后端从SQLite更换为MySQL
2. 可选启用Redis缓存
3. 提供数据迁移脚本
4. 优化数据库查询性能
5. 增加数据库连接池管理
## 前置要求
1. Python 3.7+
2. MySQL/MariaDB 服务器
3. Redis服务器 (可选)
4. 安装依赖:`pip install -r requirements.txt`
## 配置说明
`config.yaml`中添加了MySQL和Redis相关配置
```yaml
# 数据库配置
database:
# SQLite配置兼容旧版本
path: "cursor.db"
pool_size: 10
# MySQL配置
host: "localhost"
port: 3306
username: "root"
password: ""
database: "cursor_register"
# 是否使用Redis缓存
use_redis: true
# Redis配置可选当use_redis为true时生效
redis:
host: "127.0.0.1"
port: 6379
password: ""
db: 0
```
## 数据迁移步骤
1. 确保MySQL服务器已启动并已创建好数据库
2. 更新`config.yaml`配置文件,设置正确的数据库连接信息
3. 运行迁移脚本:`python migrate_db.py`
4. 迁移脚本会自动创建表结构并将旧数据导入MySQL
## 使用Redis缓存
若要启用Redis缓存以提高性能
1. 安装Redis服务器
- Windows: 使用WSL或[Windows版Redis](https://github.com/microsoftarchive/redis/releases)
- Linux: `sudo apt install redis-server` (Ubuntu) 或 `sudo yum install redis` (CentOS)
- macOS: `brew install redis`
2.`config.yaml`中设置`use_redis: true`并配置Redis连接信息
3. 确保安装了`aioredis`包:`pip install aioredis>=2.0.0`
## 常见问题
1. **无法连接到MySQL**
- 确认MySQL服务已启动
- 检查用户名和密码是否正确
- 确认数据库是否已创建
- 检查防火墙设置
2. **无法连接到Redis**
- 确认Redis服务已启动
- 检查端口和密码设置
- 如不需要Redis可设置`use_redis: false`
3. **迁移失败**
- 检查原SQLite数据库文件是否存在且有效
- 确认MySQL用户有创建表和写入数据的权限
## 性能调优
1. 优化MySQL配置
```ini
[mysqld]
innodb_buffer_pool_size = 128M
innodb_log_file_size = 32M
max_connections = 100
```
2. 优化Redis配置
```
maxmemory 128mb
maxmemory-policy allkeys-lru
```
3. 优化连接池大小:根据并发需要调整`pool_size`参数
## 注意事项
- 请确保定期备份MySQL数据库
- Redis仅用于缓存断电或重启会丢失缓存数据
- 如在多机部署,需确保时区配置一致

View File

@@ -6,12 +6,31 @@ global:
# 数据库配置
database:
# SQLite配置兼容旧版本
path: "cursor.db"
pool_size: 10
# MySQL配置
host: "localhost"
port: 3306
username: "auto_cursor_reg"
password: "this_password_jiaqiao"
database: "auto_cursor_reg"
# 是否使用Redis缓存
# 如果使用Python 3.12请确保安装redis>=4.2.0而不是aioredis
use_redis: true
# Redis配置可选当use_redis为true时生效
redis:
host: "127.0.0.1"
port: 6379
password: ""
db: 0
# 代理配置
proxy:
api_url: "https://api.proxy.com/getProxy"
api_url: "https://share.proxy.qg.net/get?key=969331C5&num=1&area=&isp=0&format=txt&seq=\r\n&distinct=false"
batch_size: 100
check_interval: 300
@@ -28,7 +47,7 @@ email:
captcha:
provider: "capsolver" # 可选值: "capsolver" 或 "yescaptcha"
capsolver:
api_key: "CAP-E0A11882290AC7ADE2F799286B8E2DA497D7CD0510BFA477F3900507809F8AA3"
api_key: "CAP-36D01B0995C7C8705DF68ACCFE4E2004FE182DDA72AC5A80F25F1E3B601C31F0"
website_url: "https://authenticator.cursor.sh"
website_key: "0x4AAAAAAAMNIvC45A4Wjjln"
yescaptcha:

View File

@@ -1,5 +1,5 @@
from dataclasses import dataclass
from typing import Tuple
from typing import Tuple, Optional
import yaml
@@ -13,8 +13,25 @@ class GlobalConfig:
@dataclass
class DatabaseConfig:
path: str
pool_size: int
# SQLite的配置字段保留用于兼容
path: Optional[str] = None
pool_size: int = 10
# MySQL配置
host: str = "localhost"
port: int = 3306
username: str = "auto_cursor_reg"
password: str = "this_password_jiaqiao"
database: str = "auto_cursor_reg"
# Redis配置
use_redis: bool = False
@dataclass
class RedisConfig:
host: str
port: int
password: str = ""
db: int = 0
@dataclass
@@ -61,29 +78,47 @@ class CaptchaConfig:
class Config:
global_config: GlobalConfig
database_config: DatabaseConfig
proxy_config: ProxyConfig
register_config: RegisterConfig
email_config: EmailConfig
captcha_config: CaptchaConfig
redis_config: Optional[RedisConfig] = None
proxy_config: ProxyConfig = None
register_config: RegisterConfig = None
email_config: EmailConfig = None
captcha_config: CaptchaConfig = None
@classmethod
def from_yaml(cls, path: str = "config.yaml"):
with open(path, 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
# 创建 database 配置对象
db_config = DatabaseConfig(**data.get('database', {}))
# 创建 redis 配置对象(如果有)
redis_config = None
if 'redis' in data and db_config.use_redis:
redis_config = RedisConfig(**data['redis'])
# 创建 captcha 配置对象
captcha_data = data['captcha']
captcha_config = CaptchaConfig(
provider=captcha_data['provider'],
capsolver=CapsolverConfig(**captcha_data['capsolver']),
yescaptcha=YesCaptchaConfig(**captcha_data['yescaptcha'])
)
captcha_data = data.get('captcha', {})
captcha_config = None
if captcha_data:
captcha_config = CaptchaConfig(
provider=captcha_data.get('provider', 'capsolver'),
capsolver=CapsolverConfig(**captcha_data.get('capsolver', {})),
yescaptcha=YesCaptchaConfig(**captcha_data.get('yescaptcha', {}))
)
# 创建其他配置对象
global_config = GlobalConfig(**data.get('global', {}))
proxy_config = ProxyConfig(**data.get('proxy', {})) if 'proxy' in data else None
register_config = RegisterConfig(**data.get('register', {})) if 'register' in data else None
email_config = EmailConfig(**data.get('email', {})) if 'email' in data else None
return cls(
global_config=GlobalConfig(**data['global']),
database_config=DatabaseConfig(**data['database']),
proxy_config=ProxyConfig(**data['proxy']),
register_config=RegisterConfig(**data['register']),
email_config=EmailConfig(**data['email']),
global_config=global_config,
database_config=db_config,
redis_config=redis_config,
proxy_config=proxy_config,
register_config=register_config,
email_config=email_config,
captcha_config=captcha_config
)

View File

@@ -1,86 +1,274 @@
import asyncio
import json
from contextlib import asynccontextmanager
from typing import Any, List, Optional
from typing import Any, Dict, List, Optional, Tuple, Union
import aiosqlite
import aiomysql
from loguru import logger
# 使用条件导入替代直接导入
REDIS_AVAILABLE = False
try:
# 尝试导入redis.asyncio (Redis-py 4.2.0+)
import redis.asyncio as redis_asyncio
REDIS_AVAILABLE = True
REDIS_TYPE = "redis-py"
except ImportError:
try:
# 尝试导入aioredis (旧版本)
import aioredis
REDIS_AVAILABLE = True
REDIS_TYPE = "aioredis"
except (ImportError, TypeError):
REDIS_AVAILABLE = False
REDIS_TYPE = None
from core.config import Config
class DatabaseManager:
def __init__(self, config: Config):
self.db_path = config.database_config.path
self._pool_size = config.database_config.pool_size
self._pool: List[aiosqlite.Connection] = []
# 数据库配置
self.db_config = config.database_config
self._pool_size = self.db_config.pool_size
self._pool = None # 连接池
self._pool_lock = asyncio.Lock()
# Redis配置
self.use_redis = self.db_config.use_redis
self.redis_config = config.redis_config if hasattr(config, 'redis_config') else None
self.redis = None
async def initialize(self):
"""初始化数据库连接池"""
logger.info("初始化数据库连接池")
async with aiosqlite.connect(self.db_path) as db:
await db.execute('''
CREATE TABLE IF NOT EXISTS email_accounts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
email TEXT UNIQUE NOT NULL,
password TEXT NOT NULL,
client_id TEXT NOT NULL,
refresh_token TEXT NOT NULL,
in_use BOOLEAN DEFAULT 0,
cursor_password TEXT,
cursor_cookie TEXT,
sold BOOLEAN DEFAULT 0,
status TEXT DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
''')
await db.commit()
# 初始化连接池
for i in range(self._pool_size):
conn = await aiosqlite.connect(self.db_path)
self._pool.append(conn)
# 创建MySQL连接池
try:
logger.info(f"连接MySQL: {self.db_config.host}:{self.db_config.port}, 用户: {self.db_config.username}, 数据库: {self.db_config.database}")
self._pool = await aiomysql.create_pool(
host=self.db_config.host,
port=self.db_config.port,
user=self.db_config.username,
password=self.db_config.password,
db=self.db_config.database,
maxsize=self._pool_size,
autocommit=True,
charset='utf8mb4'
)
logger.info("MySQL连接池创建成功")
except Exception as e:
logger.error(f"MySQL连接池创建失败: {str(e)}")
logger.error("请检查MySQL配置是否正确以及MySQL服务是否已启动")
logger.info(f"您可能需要创建MySQL用户和数据库")
logger.info(f" CREATE USER '{self.db_config.username}'@'localhost' IDENTIFIED BY '{self.db_config.password}';")
logger.info(f" CREATE DATABASE {self.db_config.database} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;")
logger.info(f" GRANT ALL PRIVILEGES ON {self.db_config.database}.* TO '{self.db_config.username}'@'localhost';")
logger.info(f" FLUSH PRIVILEGES;")
raise
# 初始化表结构
async with self.get_connection() as conn:
async with conn.cursor() as cursor:
await cursor.execute('''
CREATE TABLE IF NOT EXISTS email_accounts (
id INT AUTO_INCREMENT PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL,
client_id VARCHAR(255) NOT NULL,
refresh_token TEXT NOT NULL,
in_use BOOLEAN DEFAULT 0,
cursor_password VARCHAR(255),
cursor_cookie TEXT,
cursor_token TEXT,
sold BOOLEAN DEFAULT 0,
status VARCHAR(20) DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_status_inuse_sold (status, in_use, sold)
)
''')
# 初始化Redis连接如果配置了
if self.use_redis and REDIS_AVAILABLE and self.redis_config:
try:
# 根据检测到的Redis库类型创建连接
if REDIS_TYPE == "redis-py":
# 使用redis.asyncio创建连接
logger.info(f"使用redis-py连接Redis: {self.redis_config.host}:{self.redis_config.port}")
self.redis = redis_asyncio.Redis(
host=self.redis_config.host,
port=self.redis_config.port,
password=self.redis_config.password or None,
db=self.redis_config.db,
decode_responses=True
)
# 测试连接
await self.redis.ping()
elif REDIS_TYPE == "aioredis":
# 使用旧版aioredis创建连接
logger.info(f"使用aioredis连接Redis: {self.redis_config.host}:{self.redis_config.port}")
self.redis = await aioredis.from_url(
f"redis://{self.redis_config.host}:{self.redis_config.port}",
password=self.redis_config.password or None,
db=self.redis_config.db,
encoding="utf-8",
decode_responses=True
)
logger.info("Redis连接初始化成功")
except Exception as e:
logger.error(f"Redis连接初始化失败: {e}")
logger.info("Redis缓存将被禁用")
self.redis = None
logger.info(f"数据库连接池初始化完成,大小: {self._pool_size}")
async def cleanup(self):
"""清理数据库连接"""
for conn in self._pool:
await conn.close()
self._pool.clear()
if self._pool:
self._pool.close()
await self._pool.wait_closed()
if self.redis:
if REDIS_TYPE == "redis-py":
await self.redis.close()
else:
await self.redis.close()
logger.info("数据库连接已清理")
@asynccontextmanager
async def get_connection(self):
"""获取数据库连接"""
async with self._pool_lock:
if not self._pool:
conn = await aiosqlite.connect(self.db_path)
else:
conn = self._pool.pop()
if self._pool is None:
raise Exception("数据库连接池未初始化")
try:
yield conn
finally:
if len(self._pool) < self._pool_size:
self._pool.append(conn)
else:
await conn.close()
async with self._pool.acquire() as conn:
try:
yield conn
finally:
pass # 连接会自动返回池中
async def execute(self, query: str, params: tuple = ()) -> Any:
"""执行SQL语句"""
async with self.get_connection() as conn:
cursor = await conn.execute(query, params)
await conn.commit()
return cursor.lastrowid
logger.debug(f"执行SQL: {query}, 参数: {params}")
try:
async with self.get_connection() as conn:
async with conn.cursor() as cursor:
await cursor.execute(query, params)
async def fetch_one(self, query: str, params: tuple = ()) -> Optional[tuple]:
# 对于INSERT语句返回最后插入的ID
if query.strip().upper().startswith("INSERT"):
return cursor.lastrowid
# 对于UPDATE/DELETE语句返回影响的行数
return cursor.rowcount
except Exception as e:
logger.error(f"SQL执行失败: {query}, 参数: {params}, 错误: {str(e)}")
raise
async def fetch_one(self, query: str, params: tuple = ()) -> Optional[Dict]:
"""查询单条记录"""
async with self.get_connection() as conn:
cursor = await conn.execute(query, params)
return await cursor.fetchone()
logger.debug(f"查询单条: {query}, 参数: {params}")
async def fetch_all(self, query: str, params: tuple = ()) -> List[tuple]:
# 尝试从Redis获取缓存
cache_key = f"db:{self._make_cache_key(query, params)}"
cached_result = await self._get_from_cache(cache_key)
if cached_result is not None:
return cached_result
try:
async with self.get_connection() as conn:
async with conn.cursor(aiomysql.DictCursor) as cursor:
await cursor.execute(query, params)
result = await cursor.fetchone()
# 缓存结果
if result and self.redis:
await self._store_in_cache(cache_key, result)
logger.debug(f"查询结果: {result}")
return result
except Exception as e:
logger.error(f"查询单条失败: {query}, 参数: {params}, 错误: {str(e)}")
raise
async def fetch_all(self, query: str, params: tuple = ()) -> List[Dict]:
"""查询多条记录"""
async with self.get_connection() as conn:
cursor = await conn.execute(query, params)
return await cursor.fetchall()
logger.debug(f"查询多条: {query}, 参数: {params}")
# 尝试从Redis获取缓存
cache_key = f"db:{self._make_cache_key(query, params)}"
cached_result = await self._get_from_cache(cache_key)
if cached_result is not None:
return cached_result
try:
async with self.get_connection() as conn:
async with conn.cursor(aiomysql.DictCursor) as cursor:
await cursor.execute(query, params)
results = await cursor.fetchall()
# 缓存结果
if results and self.redis:
await self._store_in_cache(cache_key, results)
logger.debug(f"查询结果数量: {len(results)}")
return results
except Exception as e:
logger.error(f"查询多条失败: {query}, 参数: {params}, 错误: {str(e)}")
raise
async def _get_from_cache(self, key: str) -> Optional[Union[Dict, List[Dict]]]:
"""从Redis缓存获取数据"""
if not self.redis:
return None
try:
cached_data = await self.redis.get(key)
if cached_data:
return json.loads(cached_data)
except Exception as e:
logger.error(f"从缓存获取数据失败: {e}")
return None
async def _store_in_cache(self, key: str, data: Union[Dict, List[Dict]], ttl: int = 300) -> bool:
"""存储数据到Redis缓存"""
if not self.redis:
return False
try:
json_data = json.dumps(data)
if REDIS_TYPE == "redis-py":
await self.redis.setex(key, ttl, json_data)
else:
await self.redis.setex(key, ttl, json_data)
return True
except Exception as e:
logger.error(f"存储数据到缓存失败: {e}")
return False
async def clear_cache(self, pattern: str = "db:*") -> int:
"""清除缓存"""
if not self.redis:
return 0
try:
if REDIS_TYPE == "redis-py":
keys = await self.redis.keys(pattern)
if not keys:
return 0
return await self.redis.delete(*keys)
else:
keys = await self.redis.keys(pattern)
if not keys:
return 0
return await self.redis.delete(*keys)
except Exception as e:
logger.error(f"清除缓存失败: {e}")
return 0
def _make_cache_key(self, query: str, params: tuple) -> str:
"""生成缓存键"""
return f"{query}:{hash(params)}"

1
database_schema.sql Normal file
View File

@@ -0,0 +1 @@

View File

@@ -1,61 +1,110 @@
import asyncio
import sys
# Windows平台特殊处理强制使用SelectorEventLoop
if sys.platform.startswith('win'):
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
import aiosqlite
from loguru import logger
from core.config import Config
from core.database import DatabaseManager
async def import_emails(config: Config, file_path: str):
"""导入邮箱账号到数据库"""
async def import_emails(config: Config, db_manager: DatabaseManager, file_path: str):
"""导入邮箱账号到MySQL数据库"""
DEFAULT_CLIENT_ID = "9e5f94bc-e8a4-4e73-b8be-63364c29d753"
async with aiosqlite.connect(config.database_config.path) as db:
# 创建表
await db.execute('''
CREATE TABLE IF NOT EXISTS email_accounts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
email TEXT UNIQUE NOT NULL,
password TEXT NOT NULL,
client_id TEXT NOT NULL,
refresh_token TEXT NOT NULL,
in_use BOOLEAN DEFAULT 0,
cursor_password TEXT,
cursor_cookie TEXT,
cursor_token TEXT,
sold BOOLEAN DEFAULT 0,
status TEXT DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
''')
# 确保数据库连接已初始化
if not db_manager._pool:
await db_manager.initialize()
# 读取文件并导入数据
count = 0
with open(file_path, 'r', encoding='utf-8') as f:
for line in f:
if line.strip():
try:
email, password, client_id, refresh_token = line.strip().split('----')
await db.execute('''
INSERT INTO email_accounts (
email, password, client_id, refresh_token, status
) VALUES (?, ?, ?, ?, 'pending')
''', (email, password, client_id, refresh_token))
count += 1
except aiosqlite.IntegrityError:
logger.warning(f"重复的邮箱: {email}")
except ValueError:
logger.error(f"无效的数据行: {line.strip()}")
# 读取文件并导入数据
count = 0
duplicate_count = 0
error_count = 0
await db.commit()
logger.success(f"成功导入 {count} 个邮箱账号")
logger.info(f"开始从 {file_path} 导入邮箱账号")
with open(file_path, 'r', encoding='utf-8') as f:
for line_num, line in enumerate(f, 1):
if not line.strip():
continue
try:
# 解析数据行
parts = line.strip().split('----')
if len(parts) < 4:
logger.error(f"{line_num}: 格式不正确,期望 'email----password----client_id----refresh_token'")
error_count += 1
continue
email, password, client_id, refresh_token = parts
# 插入数据库
insert_query = '''
INSERT INTO email_accounts
(email, password, client_id, refresh_token, status)
VALUES (%s, %s, %s, %s, 'pending')
'''
try:
await db_manager.execute(insert_query, (email, password, client_id, refresh_token))
count += 1
if count % 100 == 0:
logger.info(f"已导入 {count} 个邮箱账号")
except Exception as e:
if "Duplicate entry" in str(e):
logger.warning(f"{line_num}: 重复的邮箱: {email}")
duplicate_count += 1
else:
logger.error(f"{line_num}: 导入失败: {str(e)}")
error_count += 1
except Exception as e:
logger.error(f"{line_num}: 处理时出错: {str(e)}")
error_count += 1
# 如果启用了Redis缓存清除相关缓存
if db_manager.redis:
cleared = await db_manager.clear_cache("db:*")
logger.info(f"已清除 {cleared} 个Redis缓存键")
logger.success(f"导入完成: 成功 {count} 个, 重复 {duplicate_count} 个, 失败 {error_count}")
return count
async def main():
config = Config.from_yaml()
await import_emails(config, "email.txt")
try:
# 加载配置
config = Config.from_yaml()
# 初始化数据库管理器
db_manager = DatabaseManager(config)
await db_manager.initialize()
# 从配置中获取邮箱文件路径,或使用默认值
file_path = config.email_config.file_path if hasattr(config, 'email_config') and config.email_config else "email.txt"
# 导入邮箱
await import_emails(config, db_manager, file_path)
except Exception as e:
logger.error(f"程序执行出错: {str(e)}")
finally:
# 清理资源
if 'db_manager' in locals():
await db_manager.cleanup()
logger.info("程序执行完毕")
if __name__ == "__main__":
# 设置日志
logger.remove()
logger.add(sys.stderr, level="INFO")
logger.add("import_emails.log", rotation="1 MB", level="DEBUG")
# 执行导入
asyncio.run(main())

327
init_database.py Normal file
View File

@@ -0,0 +1,327 @@
#!/usr/bin/env python3
"""
数据库初始化脚本
用于自动创建MySQL数据库、用户和表结构并配置Redis
"""
import os
import sys
import getpass
import asyncio
import yaml
import pymysql
import redis
from loguru import logger
# 默认配置
DEFAULT_CONFIG = {
"database": {
"host": "localhost",
"port": 3306,
"username": "auto_cursor_reg",
"password": "auto_cursor_pass",
"database": "auto_cursor_reg",
"pool_size": 10,
"use_redis": True
},
"redis": {
"host": "127.0.0.1",
"port": 6379,
"password": "",
"db": 0
}
}
# 数据库表结构
EMAIL_ACCOUNTS_TABLE = '''
CREATE TABLE IF NOT EXISTS email_accounts (
id INT AUTO_INCREMENT PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL,
client_id VARCHAR(255) NOT NULL,
refresh_token TEXT NOT NULL,
in_use BOOLEAN DEFAULT 0,
cursor_password VARCHAR(255),
cursor_cookie TEXT,
cursor_token TEXT,
sold BOOLEAN DEFAULT 0,
status VARCHAR(20) DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_status_inuse_sold (status, in_use, sold)
)
'''
def setup_logger():
"""设置日志"""
logger.remove()
logger.add(sys.stderr, level="INFO")
logger.add("init_database.log", rotation="1 MB", level="DEBUG")
def get_mysql_root_credentials():
"""获取MySQL root用户凭据"""
print("\n===== MySQL配置 =====")
print("请输入MySQL root用户凭据以创建数据库和用户")
mysql_host = input("MySQL 主机地址 [localhost]: ") or "localhost"
mysql_port = input("MySQL 端口 [3306]: ") or "3306"
try:
mysql_port = int(mysql_port)
except ValueError:
mysql_port = 3306
print("端口无效,使用默认端口 3306")
mysql_root = input("MySQL root用户名 [root]: ") or "root"
mysql_root_password = getpass.getpass("MySQL root密码: ")
return {
"host": mysql_host,
"port": mysql_port,
"user": mysql_root,
"password": mysql_root_password
}
def get_app_database_config():
"""获取应用数据库配置"""
print("\n请设置应用程序的数据库配置:")
db_name = input("数据库名称 [auto_cursor_reg]: ") or "auto_cursor_reg"
db_user = input("数据库用户名 [auto_cursor_reg]: ") or "auto_cursor_reg"
db_pass = getpass.getpass(f"为用户 {db_user} 设置密码 [auto_cursor_pass]: ")
if not db_pass:
db_pass = "auto_cursor_pass"
return {
"database": db_name,
"username": db_user,
"password": db_pass
}
def get_redis_config():
"""获取Redis配置"""
use_redis = input("\n是否使用Redis缓存? (y/n) [y]: ").lower() != "n"
if not use_redis:
return {"use_redis": False}
print("\n===== Redis配置 =====")
redis_host = input("Redis 主机地址 [127.0.0.1]: ") or "127.0.0.1"
redis_port = input("Redis 端口 [6379]: ") or "6379"
try:
redis_port = int(redis_port)
except ValueError:
redis_port = 6379
print("端口无效,使用默认端口 6379")
redis_password = getpass.getpass("Redis 密码 (如无密码请直接回车): ")
redis_db = input("Redis 数据库索引 [0]: ") or "0"
try:
redis_db = int(redis_db)
except ValueError:
redis_db = 0
print("数据库索引无效,使用默认值 0")
return {
"use_redis": True,
"redis": {
"host": redis_host,
"port": redis_port,
"password": redis_password,
"db": redis_db
}
}
def test_mysql_connection(config):
"""测试MySQL连接"""
try:
conn = pymysql.connect(**config)
conn.close()
return True
except Exception as e:
logger.error(f"MySQL连接失败: {str(e)}")
return False
def test_redis_connection(config):
"""测试Redis连接"""
try:
r = redis.Redis(
host=config["host"],
port=config["port"],
password=config["password"] if config["password"] else None,
db=config["db"]
)
r.ping()
r.close()
return True
except Exception as e:
logger.error(f"Redis连接失败: {str(e)}")
return False
def setup_mysql(root_config, app_config):
"""设置MySQL数据库和用户"""
logger.info("开始设置MySQL数据库...")
try:
# 连接到MySQL
conn = pymysql.connect(**root_config)
cursor = conn.cursor()
db_name = app_config["database"]
db_user = app_config["username"]
db_pass = app_config["password"]
# 创建数据库
logger.info(f"创建数据库: {db_name}")
cursor.execute(f"CREATE DATABASE IF NOT EXISTS `{db_name}` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci")
# 创建用户并授权
logger.info(f"创建用户: {db_user}")
# 检查用户是否已存在
cursor.execute(f"SELECT User FROM mysql.user WHERE User = '{db_user}'")
user_exists = cursor.fetchone()
if user_exists:
logger.info(f"用户 {db_user} 已存在,更新密码")
cursor.execute(f"ALTER USER '{db_user}'@'localhost' IDENTIFIED BY '{db_pass}'")
cursor.execute(f"ALTER USER '{db_user}'@'%' IDENTIFIED BY '{db_pass}'")
else:
# 创建用户 (同时创建本地和远程连接权限)
try:
cursor.execute(f"CREATE USER '{db_user}'@'localhost' IDENTIFIED BY '{db_pass}'")
cursor.execute(f"CREATE USER '{db_user}'@'%' IDENTIFIED BY '{db_pass}'")
except pymysql.err.MySQLError as e:
logger.warning(f"创建用户时出现警告 (可能用户已存在): {str(e)}")
# 授权
cursor.execute(f"GRANT ALL PRIVILEGES ON `{db_name}`.* TO '{db_user}'@'localhost'")
cursor.execute(f"GRANT ALL PRIVILEGES ON `{db_name}`.* TO '{db_user}'@'%'")
cursor.execute("FLUSH PRIVILEGES")
# 切换到新创建的数据库
cursor.execute(f"USE `{db_name}`")
# 创建表
logger.info("创建数据表: email_accounts")
cursor.execute(EMAIL_ACCOUNTS_TABLE)
conn.commit()
cursor.close()
conn.close()
logger.success("MySQL数据库设置成功")
return True
except Exception as e:
logger.error(f"设置MySQL失败: {str(e)}")
return False
def update_config_file(db_config, redis_config=None):
"""更新配置文件"""
logger.info("更新配置文件...")
config_file = "config.yaml"
try:
# 读取现有配置
with open(config_file, "r", encoding="utf-8") as f:
config = yaml.safe_load(f)
# 更新数据库配置
config["database"].update({
"host": db_config.get("host", "localhost"),
"port": db_config.get("port", 3306),
"username": db_config["username"],
"password": db_config["password"],
"database": db_config["database"],
"use_redis": db_config.get("use_redis", False)
})
# 如果启用Redis更新Redis配置
if redis_config and db_config.get("use_redis"):
config["redis"] = redis_config
# 备份原配置文件
if os.path.exists(config_file):
os.rename(config_file, f"{config_file}.bak")
logger.info(f"已备份原配置文件为 {config_file}.bak")
# 写入新配置
with open(config_file, "w", encoding="utf-8") as f:
yaml.dump(config, f, default_flow_style=False, allow_unicode=True)
logger.success(f"配置文件已更新: {config_file}")
return True
except Exception as e:
logger.error(f"更新配置文件失败: {str(e)}")
return False
def main():
"""主函数"""
setup_logger()
logger.info("开始初始化数据库")
# 获取MySQL root凭据
root_config = get_mysql_root_credentials()
# 测试MySQL连接
if not test_mysql_connection(root_config):
logger.error("无法连接到MySQL请检查凭据和服务状态")
return
# 获取应用数据库配置
db_config = get_app_database_config()
# 获取Redis配置
redis_info = get_redis_config()
db_config["use_redis"] = redis_info["use_redis"]
# 如果启用了Redis测试连接
if redis_info["use_redis"]:
redis_config = redis_info["redis"]
if not test_redis_connection(redis_config):
use_anyway = input("Redis连接测试失败是否继续? (y/n) [n]: ").lower() == "y"
if not use_anyway:
logger.warning("用户取消了初始化")
return
else:
redis_config = None
# 设置MySQL
if not setup_mysql(root_config, db_config):
return
# 合并配置
final_db_config = {
"host": root_config["host"],
"port": root_config["port"],
"username": db_config["username"],
"password": db_config["password"],
"database": db_config["database"],
"use_redis": db_config["use_redis"]
}
# 更新配置文件
update_config_file(final_db_config, redis_config)
logger.success("数据库初始化完成!")
print("\n===== 初始化完成 =====")
print("您现在可以运行以下命令导入邮箱账号:")
print(" python import_emails.py")
print("\n然后运行主程序开始注册:")
print(" python main.py")
if __name__ == "__main__":
main()

177
migrate_db.py Normal file
View File

@@ -0,0 +1,177 @@
#!/usr/bin/env python3
"""
SQLite 到 MySQL 数据迁移脚本
用于将旧的 SQLite 数据库迁移到新的 MySQL 数据库
"""
import asyncio
import sqlite3
import sys
from typing import List, Dict, Any
import aiomysql
import yaml
from loguru import logger
async def migrate_data():
"""迁移数据主函数"""
# 读取配置文件
logger.info("读取配置文件")
with open("config.yaml", "r", encoding="utf-8") as f:
config = yaml.safe_load(f)
# 获取数据库配置
db_config = config.get("database", {})
sqlite_path = db_config.get("path", "cursor.db")
mysql_config = {
"host": db_config.get("host", "localhost"),
"port": db_config.get("port", 3306),
"user": db_config.get("username", "auto_cursor_reg"), # 使用正确的默认用户名
"password": db_config.get("password", "this_password_jiaqiao"), # 使用正确的默认密码
"db": db_config.get("database", "auto_cursor_reg"), # 使用正确的默认数据库名
"charset": "utf8mb4"
}
logger.info(f"MySQL配置: 主机={mysql_config['host']}:{mysql_config['port']}, 用户={mysql_config['user']}, 数据库={mysql_config['db']}")
# 连接SQLite数据库
logger.info(f"连接SQLite数据库: {sqlite_path}")
sqlite_conn = None
mysql_conn = None
try:
sqlite_conn = sqlite3.connect(sqlite_path)
sqlite_conn.row_factory = sqlite3.Row # 启用字典行工厂
except Exception as e:
logger.error(f"无法连接SQLite数据库: {e}")
return
# 连接MySQL数据库
logger.info(f"尝试连接MySQL数据库...")
try:
mysql_conn = await aiomysql.connect(**mysql_config)
logger.info("MySQL数据库连接成功")
except Exception as e:
logger.error(f"无法连接MySQL数据库: {e}")
logger.error(f"请确认MySQL服务已启动且用户名密码正确")
logger.info(f"您可能需要创建MySQL用户和数据库")
logger.info(f" CREATE USER '{mysql_config['user']}'@'localhost' IDENTIFIED BY '{mysql_config['password']}';")
logger.info(f" CREATE DATABASE {mysql_config['db']} CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;")
logger.info(f" GRANT ALL PRIVILEGES ON {mysql_config['db']}.* TO '{mysql_config['user']}'@'localhost';")
logger.info(f" FLUSH PRIVILEGES;")
if sqlite_conn:
sqlite_conn.close()
return
try:
# 检查email_accounts表是否存在
logger.info("检查并创建MySQL表结构")
async with mysql_conn.cursor() as cursor:
await cursor.execute('''
CREATE TABLE IF NOT EXISTS email_accounts (
id INT AUTO_INCREMENT PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL,
client_id VARCHAR(255) NOT NULL,
refresh_token TEXT NOT NULL,
in_use BOOLEAN DEFAULT 0,
cursor_password VARCHAR(255),
cursor_cookie TEXT,
cursor_token TEXT,
sold BOOLEAN DEFAULT 0,
status VARCHAR(20) DEFAULT 'pending',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_status_inuse_sold (status, in_use, sold)
)
''')
await mysql_conn.commit()
# 从SQLite读取数据
logger.info("从SQLite读取数据")
try:
sqlite_cursor = sqlite_conn.cursor()
sqlite_cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='email_accounts'")
if not sqlite_cursor.fetchone():
logger.warning("SQLite数据库中不存在email_accounts表无需迁移")
return
# 获取表结构信息
sqlite_cursor.execute("PRAGMA table_info(email_accounts)")
columns_info = sqlite_cursor.fetchall()
column_names = [col["name"] for col in columns_info]
# 读取所有数据
sqlite_cursor.execute("SELECT * FROM email_accounts")
rows = sqlite_cursor.fetchall()
logger.info(f"从SQLite读取到 {len(rows)} 条数据")
except Exception as e:
logger.error(f"读取SQLite数据失败: {e}")
return
# 迁移数据到MySQL
if rows:
logger.info("开始迁移数据到MySQL")
try:
async with mysql_conn.cursor() as cursor:
# 检查MySQL表中是否已有数据
await cursor.execute("SELECT COUNT(*) FROM email_accounts")
result = await cursor.fetchone()
if result and result[0] > 0:
logger.warning("MySQL表中已存在数据是否继续(y/n)")
response = input().strip().lower()
if response != 'y':
logger.info("用户取消迁移")
return
# 构建插入查询
placeholders = ", ".join(["%s"] * len(column_names))
columns = ", ".join(column_names)
query = f"INSERT INTO email_accounts ({columns}) VALUES ({placeholders}) ON DUPLICATE KEY UPDATE id=id"
# 批量插入数据
batch_size = 100
for i in range(0, len(rows), batch_size):
batch = rows[i:i+batch_size]
batch_values = []
for row in batch:
# 将行数据转换为列表
row_values = [row[name] for name in column_names]
batch_values.append(row_values)
await cursor.executemany(query, batch_values)
await mysql_conn.commit()
logger.info(f"已迁移 {min(i+batch_size, len(rows))}/{len(rows)} 条数据")
logger.success(f"数据迁移完成,共迁移 {len(rows)} 条数据")
except Exception as e:
logger.error(f"迁移数据到MySQL失败: {e}")
else:
logger.warning("SQLite数据库中没有数据需要迁移")
finally:
# 安全关闭连接
if sqlite_conn:
sqlite_conn.close()
logger.debug("SQLite连接已关闭")
if mysql_conn:
await mysql_conn.close()
logger.debug("MySQL连接已关闭")
if __name__ == "__main__":
# Windows平台特殊处理强制使用SelectorEventLoop
if sys.platform.startswith('win'):
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
# 设置日志
logger.remove()
logger.add(sys.stderr, level="INFO")
logger.add("migrate.log", rotation="1 MB", level="DEBUG")
# 执行迁移
logger.info("开始执行数据迁移")
asyncio.run(migrate_data())
logger.info("数据迁移脚本执行完毕")

View File

@@ -22,6 +22,13 @@ python-dateutil
# Database
aiosqlite
aiomysql>=0.1.1
pymysql>=1.0.2
# Redis - 使用其中一个
# 如果使用Python 3.12推荐使用redis-py而不是aioredis
redis>=4.2.0
# aioredis>=2.0.0 # 不建议在Python 3.12上使用
# Logging
loguru==0.7.2

View File

@@ -39,29 +39,44 @@ class EmailManager:
"""批量获取未使用的邮箱账号"""
logger.info(f"尝试获取 {num} 个未使用的邮箱账号")
query = '''
UPDATE email_accounts
SET in_use = 1, updated_at = CURRENT_TIMESTAMP
WHERE id IN (
SELECT id FROM email_accounts
WHERE in_use = 0 AND sold = 0 AND status = 'pending'
LIMIT ?
)
RETURNING id, email, password, client_id, refresh_token
'''
# 1. 先查询符合条件的账号ID列表
select_query = """
SELECT id, email, password, client_id, refresh_token
FROM email_accounts
WHERE in_use = 0 AND sold = 0 AND status = 'pending'
LIMIT %s
"""
accounts = await self.db.fetch_all(select_query, (num,))
results = await self.db.fetch_all(query, (num,))
logger.debug(f"实际获取到 {len(results)}账号")
if not accounts:
logger.debug("没有找到符合条件的账号")
return []
# 2. 提取账号ID列表
account_ids = [account['id'] for account in accounts]
# 3. 更新这些账号的状态
if account_ids:
placeholders = ', '.join(['%s' for _ in account_ids])
update_query = f"""
UPDATE email_accounts
SET in_use = 1, updated_at = CURRENT_TIMESTAMP
WHERE id IN ({placeholders})
"""
await self.db.execute(update_query, tuple(account_ids))
# 4. 返回账号数据
logger.debug(f"实际获取到 {len(accounts)} 个账号")
return [
EmailAccount(
id=row[0],
email=row[1],
password=row[2],
client_id=row[3],
refresh_token=row[4],
id=row['id'],
email=row['email'],
password=row['password'],
client_id=row['client_id'],
refresh_token=row['refresh_token'],
in_use=True
)
for row in results
for row in accounts
]
async def update_account_status(self, account_id: int, status: str):
@@ -69,10 +84,10 @@ class EmailManager:
query = '''
UPDATE email_accounts
SET
status = ?,
status = %s,
in_use = 0,
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
WHERE id = %s
'''
await self.db.execute(query, (status, account_id))
@@ -81,14 +96,14 @@ class EmailManager:
query = '''
UPDATE email_accounts
SET
cursor_password = ?,
cursor_cookie = ?,
cursor_token = ?,
cursor_password = %s,
cursor_cookie = %s,
cursor_token = %s,
in_use = 0,
sold = 1,
status = 'success',
updated_at = CURRENT_TIMESTAMP
WHERE id = ?
WHERE id = %s
'''
await self.db.execute(query, (cursor_password, cursor_cookie, cursor_token, account_id))
@@ -97,7 +112,7 @@ class EmailManager:
query = '''
UPDATE email_accounts
SET in_use = 0, updated_at = CURRENT_TIMESTAMP
WHERE id = ?
WHERE id = %s
'''
await self.db.execute(query, (account_id,))
@@ -236,10 +251,10 @@ class EmailManager:
logger.debug(f"从文本中提取到验证码: {code}")
return code
logger.warning(f"[{email}] 未能从邮件中提取到验证码")
logger.debug(f"[{email}] 邮件内容预览: " + body[:200])
logger.warning(f"未能从邮件中提取到验证码")
logger.debug(f"邮件内容预览: " + body[:200])
return None
except Exception as e:
logger.error(f"[{email}] 提取验证码失败: {str(e)}")
logger.error(f"提取验证码失败: {str(e)}")
return None

View File

@@ -14,7 +14,7 @@ class ProxyPool:
async def batch_get(self, num: int) -> List[str]:
"""获取num个代理"""
# 临时代理
return ['http://127.0.0.1:3057'] * num
return ['http://60.188.79.110:20051'] * num
try:
response = await self.fetch_manager.request(