-
ORM增删改查并发性能测试
博客中EFCore测试结果比其它ORM快的原因
涉及博客《ORM增删改查并发性能测试》《ORM增删改查并发性能测试2》《.NET 6 EFCore WebApi 使用 JMeter 进行吞吐量测试》
测试代码中EFCore的多字段排序写法不正确,导致生成的SQL不正确,SQL中只有Id排序,没有CreateTime字段的排序!
EFCore不正确的多字段排序写法
string remark = "测试";
int id = 20;
var list = await _db.Set<SysUser>().AsNoTracking()
.Where(t => t.Id > id && t.RealName.Contains(remark))
.OrderByDescending(t => t.CreateTime)
.OrderBy(t => t.Id).Skip(1800).Take(200).ToListAsync();
EFCore正确的多字段排序写法
string remark = "测试";
int id = 20;
var list = await _db.Set<SysUser>().AsNoTracking()
.Where(t => t.Id > id && t.RealName.Contains(remark))
.OrderByDescending(t => t.CreateTime)
.ThenBy(t => t.Id).Skip(1800).Take(200).ToListAsync();
重新测试EFCore
测试命令:
第一轮:
跟其它ORM差不多。
第二轮:
跟其它ORM差不多。
代码:
ORM增删改查并发性能测试
测试的现实意义
这两天在对一些ORM进行性能测试(涉及SqlSugar、FreeSql、Fast.Framework、Dapper.LiteSql),测试用的是Winform程序,别人第一眼看到我的程序,说,你这测试没意义!
可能我的测试程序的某些地方写的比较变态吧,但我认为有现实意义,并且网上有相关网站崩溃问题的文章,那什么“爆高分析”,“崩溃问题”,WinDbg用的很6,那写程序阶段为什么没发现呢?
场景
假设WebApi(或者说网站后台服务)正在被高并发访问,一段时间后,服务的线程数量达到了200正常的是类似这样的,任务来了,就干活:及以上,那么.NET的线程池,如果线程处于空闲状态,默认是20秒后释放线程,假设这200多个线程空闲了,但是还没有达到空闲20秒的时间,还没有释放,如果此时,又有大量高并发的访问,200个线程去操作数据库,那就可能很危险了。
关于性能
虽然代码写的可能有点变态,但我的测试不是很严格,也不是很规范。
性能如果不是数量级的差,问题都不大,比如一个是0.8秒,一个是1.8秒,那1.8秒的慢是慢了一点,但问题不大;但如果一个是0.8秒,一个是10秒,那10秒的这个,可能就有点问题了,即使别人用的Emit你用的反射,也不应该差这么多。
当然了,我的测试,数据规模不大,10000条,button可能会点10次,那也才10万数据量,在数据量方面我没有做极端测试,我的重点不是这个,因为很少会去查100万数据到内存中,就暂不讨论这个。
这个测试除了大体上测试一下性能,主要就是增加了并发测试。
非并发性能测试的截图我就不放全了,这几个ORM有差别,但我觉得2秒3秒还是6秒甚至8秒,都问题不大的,都算差不多,都算堪用。
测试代码
测试数据库是MySQL版本5.7.28,MySql的连接池大小是1200,MySql.Data.dll目前用的是最新的8.0.30版本(题外话,它里面的异步是假异步,把同步方法包装成的异步,我说怎么反而慢了呢,真异步要使用MySqlConnector.dll,Fast.Framework使用的是这个性能很好)。
1. 数据库连接字符串
server=localhost;database=sqlsugar_test;user id=root;password=123456;character set=utf8mb4;
这里没有设置min pool size和max pool size,也没有设置connection timeout,没有设置的话connection timeout默认是15秒。
2. Winfrom配置
在写Winform做高并发测试的时候,有个参数可能需要设置
<system.net>
<connectionManagement>
<add address="*" maxconnection="1200" />
</connectionManagement>
</system.net>
测试高并发网络请求的时候,要设置这个,那么数据库增删改查要不要设置呢?反正就一个参数,加或不加,可以试试。
3. 实体类
以SqlSugar的为例吧,几种ORM测试用的表结构是相同的。
using System;
using System.Linq;
using System.Text;
using SqlSugar;
namespace Models
{
///<summary>
///用户表
///</summary>
[SugarTable("sys_user")]
public partial class SysUser
{
public SysUser(){
}
/// <summary>
/// Desc:主键
/// Default:
/// Nullable:False
/// </summary>
[SugarColumn(IsPrimaryKey=true,IsIdentity=true,ColumnName="id")]
public long Id {get;set;}
/// <summary>
/// Desc:用户名
/// Default:
/// Nullable:False
/// </summary>
[SugarColumn(ColumnName="user_name")]
public string UserName {get;set;}
/// <summary>
/// Desc:用户姓名
/// Default:
/// Nullable:True
/// </summary>
[SugarColumn(ColumnName="real_name")]
public string RealName {get;set;}
/// <summary>
/// Desc:用户密码
/// Default:
/// Nullable:False
/// </summary>
[SugarColumn(ColumnName="password")]
public string Password {get;set;}
/// <summary>
/// Desc:备注
/// Default:
/// Nullable:True
/// </summary>
[SugarColumn(ColumnName="remark")]
public string Remark {get;set;}
/// <summary>
/// Desc:创建者ID
/// Default:
/// Nullable:False
/// </summary>
[SugarColumn(ColumnName="create_userid")]
public string CreateUserid {get;set;}
/// <summary>
/// Desc:创建时间
/// Default:
/// Nullable:False
/// </summary>
[SugarColumn(ColumnName="create_time")]
public DateTime CreateTime {get;set;}
/// <summary>
/// Desc:更新者ID
/// Default:
/// Nullable:True
/// </summary>
[SugarColumn(ColumnName="update_userid")]
public string UpdateUserid {get;set;}
/// <summary>
/// Desc:更新时间
/// Default:
/// Nullable:True
/// </summary>
[SugarColumn(ColumnName="update_time")]
public DateTime? UpdateTime {get;set;}
}
}
4. 测试界面,感受一下
5. FreeSql测试代码
public class FreeSqlUtil
{
#region CreateFreeSqlClient
public static IFreeSql CreateFreeSqlClient()
{
IFreeSql db = new FreeSql.FreeSqlBuilder()
.UseConnectionString(FreeSql.DataType.MySql, @"server=localhost;database=freesql_test;user id=root;password=123456;character set=utf8mb4;")
.UseGenerateCommandParameterWithLambda(true)
.Build(); //请务必定义成 Singleton 单例模式
return db;
}
#endregion
}
using Models;
using NLog;
using FreeSqlDemo.Utils;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
using FreeSql.Aop;
using Newtonsoft.Json;
using System.Threading;
namespace FreeSqlDemo
{
public partial class Form1 : Form
{
#region 变量
private Logger _log = NLog.LogManager.GetCurrentClassLogger();
private IFreeSql _db;
private int _count = 10000;
private bool _printSql = false;
#endregion
#region Form1
public Form1()
{
InitializeComponent();
}
#endregion
#region Form1_Load
private void Form1_Load(object sender, EventArgs e)
{
_db = FreeSqlUtil.CreateFreeSqlClient();
_db.Aop.CurdAfter += CurdAfter;
_printSql = cbxPrintSql.Checked;
RunTask(() =>
{
_db.Queryable<SysUser>().Count(); //预热
Log("预热完成");
});
}
#endregion
#region Log
private void Log(string log)
{
if (!this.IsDisposed)
{
string msg = DateTime.Now.ToString("mm:ss.fff") + " " + log + "\r\n\r\n";
if (this.InvokeRequired)
{
this.BeginInvoke(new Action(() =>
{
textBox1.AppendText(msg);
}));
}
else
{
textBox1.AppendText(msg);
}
}
}
#endregion
#region 清空输出框
private void button10_Click(object sender, EventArgs e)
{
textBox1.Text = string.Empty;
}
#endregion
#region RunTask
private Task RunTask(Action action)
{
return Task.Run(() =>
{
try
{
action();
}
catch (Exception ex)
{
Log(ex.Message + "\r\n" + ex.StackTrace);
}
});
}
private Task RunTask<T>(Action<T> action, T t)
{
return Task.Factory.StartNew(obj =>
{
try
{
action((T)obj);
}
catch (Exception ex)
{
Log(ex.ToString());
throw;
}
}, t);
}
#endregion
#region 打印SQL
private void CurdAfter(object sender, CurdAfterEventArgs e)
{
if (_printSql)
{
RunTask(() =>
{
string msg = "SQL:" + e.Sql + "\r\nParam:" + JsonConvert.SerializeObject(e.DbParms.ToDictionary(it => it.ParameterName, it => it.Value));
Console.WriteLine(msg);
_log.Debug(msg);
});
}
}
#endregion
#region cbxPrintSql_Click
private void cbxPrintSql_Click(object sender, EventArgs e)
{
_printSql = cbxPrintSql.Checked;
}
#endregion
#region 生成实体类
private void button1_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("开始生成实体类");
Log("生成实体类完成");
});
}
#endregion
#region 测试查询
private void button2_Click(object sender, EventArgs e)
{
Log("开始查询");
SysUser conditionModel = new SysUser();
conditionModel.Remark = "管理员";
string remark = "管理员";
List<SysUser> list = _db.Queryable<SysUser>().Where(t => t.Id < 20 && t.Remark.Contains(conditionModel.Remark) && t.CreateTime > new DateTime(2010, 1, 1)).ToList();
foreach (SysUser user in list)
{
Log(string.Format("{0},{1},{2},{3}", user.UserName, user.RealName, user.Remark, user.CreateTime.ToString("yyyy-MM-dd")));
}
Log("查询结束 count=" + list.Count);
}
#endregion
#region 测试批量修改
private void button3_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Random rnd = new Random();
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20).ToList();
Log("批量修改 开始 count=" + userList.Count.ToString("0 000"));
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + rnd.Next(0, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
DateTime dt = DateTime.Now;
_db.Ado.Transaction(() =>
{
_db.Update<SysUser>().SetSource(userList).ExecuteAffrows();
});
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试批量添加
private void button4_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
userList.Add(user);
}
Log("批量添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
_db.Ado.Transaction(() =>
{
_db.Insert(userList).ExecuteAffrows();
});
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量添加 结束,完成:" + time + "秒");
});
}
#endregion
#region 删除
private void button5_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("删除 开始");
_db.Delete<SysUser>().Where(t => t.Id > 20).ExecuteAffrows();
Log("删除 完成");
});
}
#endregion
#region 测试循环修改
private void button7_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Random rnd = new Random();
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20).ToList();
Log("循环修改 开始 count=" + userList.Count.ToString("0 000"));
DateTime dt = DateTime.Now;
_db.Ado.Transaction(() =>
{
var repo = _db.GetRepository<SysUser>();
foreach (SysUser user in userList)
{
repo.Attach(user);
}
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + rnd.Next(0, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
foreach (SysUser user in userList)
{
repo.Update(user);
}
});
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试循环添加
private void button6_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
userList.Add(user);
}
Log("循环添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
_db.Ado.Transaction(() =>
{
foreach (SysUser user in userList)
{
_db.Insert(user).ExecuteIdentity();
}
});
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 查询
private void button9_Click(object sender, EventArgs e)
{
RunTask(() =>
{
_db.Queryable<SysUser>().Where(t => t.Id == 1).ToList(); //预热
Log("查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20 && t.RealName.Contains("测试")).OrderByDescending(t => t.CreateTime).OrderBy(t => t.Id).ToList();
Log("查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 分页查询
private void button8_Click(object sender, EventArgs e)
{
RunTask(() =>
{
_db.Queryable<SysUser>().Where(t => t.Id == 1).ToList(); //预热
Log("分页查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
long total = _db.Queryable<SysUser>().Count();
int pageSize = 100;
int pageCount = (int)((total - 1) / pageSize + 1);
List<SysUser> userList = new List<SysUser>();
for (int page = 1; page <= pageCount; page++)
{
userList.AddRange(_db.Queryable<SysUser>().Where(t => t.Id > 20 && t.RealName.Contains("测试")).OrderByDescending(t => t.CreateTime).OrderBy(t => t.Id).Page(page, pageSize).ToList());
}
Log("分页查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("分页查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发查询
private void button11_Click(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
RunTask(() =>
{
Log("并发查询 开始");
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
for (int i = 0; i < 200; i++)
{
Task task = RunTask(() =>
{
List<SysUser> userList = _db.Queryable<SysUser>()
.Where(t => t.Id > 20 && t.RealName.Contains("测试"))
.OrderByDescending(t => t.CreateTime).OrderBy(t => t.Id).ToList();
Log("查询结果 count=" + userList.Count.ToString());
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发插入
private void button12_Click(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Remark = "测试插入用户" + i;
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("并发插入 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = RunTask(user =>
{
_db.Insert(user).ExecuteIdentity();
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发插入 完成,耗时:" + time + "秒");
});
}
#endregion
}
}
6. SqlSugar测试代码
public class SqlSugarUtil
{
public static readonly string ConnectionString = "server=localhost;database=sqlsugar_test;user id=root;password=123456;character set=utf8mb4;";
#region CreateSqlSugarClient
public static SqlSugarScope CreateSqlSugarClient()
{
SqlSugarScope db = new SqlSugarScope(new ConnectionConfig()
{
ConnectionString = ConnectionString,//连接符字串
DbType = SqlSugar.DbType.MySql, //数据库类型
IsAutoCloseConnection = true //不设成true要手动close
});
return db;
}
#endregion
#region CreateModels 生成实体类
public static void CreateModels(SqlSugarClient db, string tableName = null)
{
......此处省略
}
#endregion
}
using Models;
using NLog;
using SqlSugar;
using SqlSugarDemo.Utils;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace SqlSugarDemo
{
public partial class Form1 : Form
{
#region 变量
private Logger _log = NLog.LogManager.GetCurrentClassLogger();
private SqlSugarScope _db;
private int _count = 10000;
#endregion
#region Form1
public Form1()
{
InitializeComponent();
}
#endregion
#region Form1_Load
private void Form1_Load(object sender, EventArgs e)
{
_db = SqlSugarUtil.CreateSqlSugarClient();
_db.Aop.OnLogExecuting = OnLogExecuting;
RunTask(() =>
{
_db.Queryable<SysUser>().Count(); //预热
Log("预热完成");
});
}
#endregion
#region Log
private void Log(string log)
{
if (!this.IsDisposed)
{
string msg = DateTime.Now.ToString("mm:ss.fff") + " " + log + "\r\n\r\n";
if (this.InvokeRequired)
{
this.BeginInvoke(new Action(() =>
{
textBox1.AppendText(msg);
}));
}
else
{
textBox1.AppendText(msg);
}
}
}
#endregion
#region 清空输出框
private void button10_Click(object sender, EventArgs e)
{
textBox1.Text = string.Empty;
}
#endregion
#region RunTask
private Task RunTask(Action action)
{
return Task.Run(() =>
{
try
{
action();
}
catch (Exception ex)
{
Log(ex.Message + "\r\n" + ex.StackTrace);
}
});
}
private Task RunTask<T>(Action<T> action, T t)
{
return Task.Factory.StartNew(obj =>
{
try
{
action((T)obj);
}
catch (Exception ex)
{
Log(ex.ToString());
}
}, t);
}
#endregion
#region 打印SQL
private void OnLogExecuting(string sql, SugarParameter[] paramArr)
{
RunTask(() =>
{
string msg = "SQL:" + sql + "\r\nParam:" + _db.Utilities.SerializeObject(paramArr.ToDictionary(it => it.ParameterName, it => it.Value));
Console.WriteLine(msg);
_log.Debug(msg);
});
}
#endregion
#region 生成实体类
private void button1_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("开始生成实体类");
SqlSugarUtil.CreateModels(new SqlSugarClient(new ConnectionConfig()
{
ConnectionString = SqlSugarUtil.ConnectionString,//连接符字串
DbType = SqlSugar.DbType.MySql, //数据库类型
IsAutoCloseConnection = true //不设成true要手动close
}));
Log("生成实体类完成");
});
}
#endregion
#region 测试查询
private void button2_Click(object sender, EventArgs e)
{
Log("开始查询");
SysUser conditionModel = new SysUser();
conditionModel.Remark = "管理员";
string remark = "管理员";
List<SysUser> list = _db.Queryable<SysUser>().Where(t => t.Id < 20 && t.Remark.Contains(conditionModel.Remark) && t.CreateTime > new DateTime(2010, 1, 1)).ToList();
foreach (SysUser user in list)
{
Log(string.Format("{0},{1},{2},{3}", user.UserName, user.RealName, user.Remark, user.CreateTime.ToString("yyyy-MM-dd")));
}
Log("查询结束 count=" + list.Count);
}
#endregion
#region 测试批量修改
private void button3_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Random rnd = new Random();
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20).ToList();
Log("批量修改 开始 count=" + userList.Count.ToString("0 000"));
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + rnd.Next(0, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
DateTime dt = DateTime.Now;
try
{
_db.Ado.BeginTran();
_db.Updateable(userList).ExecuteCommand();
_db.Ado.CommitTran();
}
catch (Exception ex)
{
_db.Ado.RollbackTran();
throw ex;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试批量添加
private void button4_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
userList.Add(user);
}
Log("批量添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
try
{
_db.Ado.BeginTran();
_db.Insertable(userList).ExecuteCommand();
_db.Ado.CommitTran();
}
catch (Exception ex)
{
_db.Ado.RollbackTran();
throw ex;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量添加 结束,完成:" + time + "秒");
});
}
#endregion
#region 删除
private void button5_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("删除 开始");
_db.Deleteable<SysUser>(t => t.Id > 20).ExecuteCommand();
Log("删除 完成");
});
}
#endregion
#region 测试循环修改
private void button7_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Random rnd = new Random();
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20).ToList();
Log("循环修改 开始 count=" + userList.Count.ToString("0 000"));
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + rnd.Next(0, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
DateTime dt = DateTime.Now;
try
{
_db.Ado.BeginTran();
foreach (SysUser user in userList)
{
_db.Updateable(user).ExecuteCommand();
}
_db.Ado.CommitTran();
}
catch (Exception ex)
{
_db.Ado.RollbackTran();
throw ex;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试循环添加
private void button6_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
userList.Add(user);
}
Log("循环添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
try
{
_db.Ado.BeginTran();
foreach (SysUser user in userList)
{
_db.Insertable(user).ExecuteCommand();
}
_db.Ado.CommitTran();
}
catch (Exception ex)
{
_db.Ado.RollbackTran();
throw ex;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 查询
private void button9_Click(object sender, EventArgs e)
{
RunTask(() =>
{
_db.Queryable<SysUser>().Where(t => t.Id == 1).ToList(); //预热
Log("查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20 && t.RealName.Contains("测试")).OrderBy(t => t.CreateTime, OrderByType.Desc).OrderBy(t => t.Id).ToList();
Log("查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 分页查询
private void button8_Click(object sender, EventArgs e)
{
RunTask(() =>
{
_db.Queryable<SysUser>().Where(t => t.Id == 1).ToList(); //预热
Log("分页查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
int total = _db.Queryable<SysUser>().Count();
int pageSize = 100;
int pageCount = (total - 1) / pageSize + 1;
List<SysUser> userList = new List<SysUser>();
for (int page = 1; page <= pageCount; page++)
{
userList.AddRange(_db.Queryable<SysUser>().Where(t => t.Id > 20 && t.RealName.Contains("测试")).OrderBy(t => t.CreateTime, OrderByType.Desc).OrderBy(t => t.Id).ToPageList(page, pageSize));
}
Log("分页查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("分页查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发查询
private void button11_Click(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
RunTask(() =>
{
Log("并发查询 开始");
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
for (int i = 0; i < 200; i++)
{
Task task = RunTask(() =>
{
List<SysUser> userList = _db.Queryable<SysUser>().Where(t => t.Id > 20 && t.RealName.Contains("测试")).OrderBy(t => t.CreateTime, OrderByType.Desc).OrderBy(t => t.Id).ToList();
Log("查询结果 count=" + userList.Count.ToString());
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发插入
private void button12_Click(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Remark = "测试插入用户" + i;
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("并发插入 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = RunTask(user =>
{
_db.Insertable(user).ExecuteCommand();
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发插入 完成,耗时:" + time + "秒");
});
}
#endregion
}
}
8. Fast.Framework测试代码
(.NET6,Dapper.LiteSql .NET Framework 4.5.2和.NET6各写了一份测试,实测插入和更新速度差不多,查询.NET6会稍微快一些)
using Fast.Framework;
using Fast.Framework.Interfaces;
using Fast.Framework.Models;
using Fast.Framework.Extensions;
using Fast.Framework.Aop;
using Models;
namespace FastFrameworkDemo
{
public partial class Form1 : Form
{
#region 变量
private Random _rnd = new Random();
private int _count = 10000;
private bool _showSqlLog = false;
#endregion
#region Form1
public Form1()
{
InitializeComponent();
}
#endregion
#region Form1_Load
private void Form1_Load(object sender, EventArgs e)
{
RunTask(async () =>
{
await GetDbContext().Query<SysUser>().CountAsync();
Log("预热完成");
});
}
#endregion
#region Log
private void Log(string log)
{
if (!this.IsDisposed)
{
string msg = DateTime.Now.ToString("mm:ss.fff") + " " + log + "\r\n\r\n";
if (this.InvokeRequired)
{
this.BeginInvoke(new Action(() =>
{
textBox1.AppendText(msg);
}));
}
else
{
textBox1.AppendText(msg);
}
}
}
#endregion
#region 清空输出框
private void button10_Click(object sender, EventArgs e)
{
textBox1.Text = string.Empty;
}
#endregion
#region RunTask
private Task RunTask(Action action)
{
return Task.Run(() =>
{
try
{
action();
}
catch (Exception ex)
{
Log(ex.ToString());
}
});
}
private Task RunTask<T>(Action<T> action, T t)
{
return Task.Factory.StartNew(obj =>
{
try
{
action((T)obj);
}
catch (Exception ex)
{
Log(ex.ToString());
}
}, t);
}
#endregion
#region GetDbContext
private IDbContext GetDbContext()
{
IDbContext _db = new DbContext(new List<DbOptions>() {
new DbOptions()
{
DbId = "1",
DbType = DbType.MySQL,
ProviderName = "MySqlConnector",
FactoryName = "MySqlConnector.MySqlConnectorFactory,MySqlConnector",
ConnectionStrings = "server=localhost;database=fast_framework_test;user id=root;password=123456;character set=utf8mb4;"
}
});
if (_showSqlLog)
{
_db.Aop.DbLog = (sql, dp) =>
{
Console.WriteLine($"执行Sql:{sql}");
if (dp != null)
{
foreach (var item in dp)
{
Console.WriteLine($"参数名称:{item.ParameterName} 参数值:{item.Value}");
}
}
};
}
return _db;
}
#endregion
#region 删除
private void button1_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
Log("删除 开始");
await GetDbContext().Delete<SysUser>().Where(t => t.Id > 20).ExceuteAsync();
Log("删除 完成");
});
}
#endregion
#region 批量修改
private void button2_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = await GetDbContext().Query<SysUser>().Where(t => t.Id > 20).ToListAsync();
Log("批量修改 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + _rnd.Next(1, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
await GetDbContext().Update(userList).ExceuteAsync();
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 批量添加
private void button3_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("批量添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
try
{
await GetDbContext().Insert(userList).ExceuteAsync();
}
catch
{
//todo:没有rollback?
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 循环修改
private void button4_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = await GetDbContext().Query<SysUser>().Where(t => t.Id > 20).ToListAsync();
Log("循环修改 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
try
{
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + _rnd.Next(1, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
var db = GetDbContext();
await db.Ado.BeginTranAsync();
foreach (SysUser user in userList)
{
await db.Update(user).ExceuteAsync();
}
await db.Ado.CommitTranAsync();
}
catch
{
//todo:没有rollback?
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 循环添加
private void button5_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("循环添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
try
{
var db = GetDbContext();
await db.Ado.BeginTranAsync();
foreach (SysUser user in userList)
{
await db.Insert(user).ExceuteAsync();
}
await db.Ado.CommitTranAsync();
}
catch
{
//todo:没有rollback?
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 查询
private void button6_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
Log("查询 开始");
DateTime dt = DateTime.Now;
var db = GetDbContext();
for (int i = 0; i < 10; i++)
{
List<SysUser> userList = await db.Query<SysUser>()
.Where(t => t.Id > 20 && t.RealName.Contains("%测试%"))
.OrderBy("create_time", "desc")
.OrderBy("id", "asc").ToListAsync();
Log("查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 分页查询
private void button7_Click(object sender, EventArgs e)
{
Log("尚未实现");
}
#endregion
#region 并发查询
private void button8_Click(object sender, EventArgs e)
{
RunTask(() =>
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
Log("并发查询 开始");
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
for (int i = 0; i < 200; i++)
{
Task task = RunTask(async () =>
{
List<SysUser> userList = await GetDbContext().Query<SysUser>()
.Where(t => t.Id > 20 && t.RealName.Contains("%测试%"))
.OrderBy("create_time", "desc")
.OrderBy("id", "asc").ToListAsync();
Log("查询结果 count=" + userList.Count.ToString());
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发插入
private void button9_Click(object sender, EventArgs e)
{
RunTask(() =>
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Remark = "测试插入用户" + i;
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("并发插入 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = RunTask(async user =>
{
await GetDbContext().Insert(user).ExceuteAsync();
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发插入 完成,耗时:" + time + "秒");
});
}
#endregion
}
}
9. Dapper.LiteSql
(还有一个不依赖Dapper的版本,那个插入和更新性能差不多,查询因为用的是反射,大约慢50%)
public class LiteSqlFactory
{
#region 变量
private static ILiteSqlClient _liteSqlClient = new LiteSqlClient(ConfigurationManager.ConnectionStrings["DefaultConnection"].ToString(), DBType.MySQL, new MySQLProvider());
#endregion
#region 获取 ISession
/// <summary>
/// 获取 ISession
/// </summary>
/// <param name="splitTableMapping">分表映射</param>
public static ISession GetSession(SplitTableMapping splitTableMapping = null)
{
return _liteSqlClient.GetSession(splitTableMapping);
}
#endregion
#region 获取 ISession (异步)
/// <summary>
/// 获取 ISession (异步)
/// </summary>
/// <param name="splitTableMapping">分表映射</param>
public static async Task<ISession> GetSessionAsync(SplitTableMapping splitTableMapping = null)
{
return await _liteSqlClient.GetSessionAsync(splitTableMapping);
}
#endregion
}
using DAL;
using Dapper.LiteSql;
using Models;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
using Utils;
namespace PerformanceTest
{
public partial class Form1 : Form
{
#region 变量
private BsOrderDal m_BsOrderDal = ServiceHelper.Get<BsOrderDal>();
private SysUserDal m_SysUserDal = ServiceHelper.Get<SysUserDal>();
private Random _rnd = new Random();
private int _count = 10000;
#endregion
#region Form1
public Form1()
{
InitializeComponent();
}
#endregion
#region Form1_Load
private void Form1_Load(object sender, EventArgs e)
{
RunTask(() =>
{
LiteSqlFactory.GetSession(); //预热
Log("预热完成");
});
}
#endregion
#region Log
private void Log(string log)
{
if (!this.IsDisposed)
{
string msg = DateTime.Now.ToString("mm:ss.fff") + " " + log + "\r\n\r\n";
if (this.InvokeRequired)
{
this.BeginInvoke(new Action(() =>
{
textBox1.AppendText(msg);
}));
}
else
{
textBox1.AppendText(msg);
}
}
}
#endregion
#region 清空输出框
private void button10_Click(object sender, EventArgs e)
{
textBox1.Text = string.Empty;
}
#endregion
#region RunTask
private Task RunTask(Action action)
{
return Task.Run(() =>
{
try
{
action();
}
catch (Exception ex)
{
Log(ex.ToString());
throw;
}
});
}
private Task RunTask<T>(Action<T> action, T t)
{
return Task.Factory.StartNew(obj =>
{
try
{
action((T)obj);
}
catch (Exception ex)
{
Log(ex.ToString());
throw;
}
}, t);
}
#endregion
#region 删除
private void button5_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("删除 开始");
var session = LiteSqlFactory.GetSession();
session.DeleteByCondition<SysUser>(string.Format("id>=12"));
Log("删除 完成");
});
}
#endregion
#region 测试批量修改
private void button3_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = m_SysUserDal.GetList("select t.* from sys_user t where t.id > 20");
Log("批量修改 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
var session = LiteSqlFactory.GetSession();
try
{
session.OnExecuting = (sql, param) =>
{
Console.WriteLine(sql); //打印SQL
};
session.AttachOld(userList);
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + _rnd.Next(1, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
userList.ForEach(item => item.UpdateTime = DateTime.Now);
session.BeginTransaction();
session.Update(userList);
session.CommitTransaction();
}
catch (Exception ex)
{
session.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试批量添加
private void button4_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("批量添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
var session = LiteSqlFactory.GetSession();
try
{
session.OnExecuting = (s, p) => Console.WriteLine(s); //打印SQL
session.BeginTransaction();
session.Insert(userList);
session.CommitTransaction();
}
catch
{
session.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试循环修改
private void button7_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = m_SysUserDal.GetList("select t.* from sys_user t where t.id > 20");
Log("循环修改 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
var session = LiteSqlFactory.GetSession();
try
{
session.OnExecuting = (sql, param) =>
{
Console.WriteLine(sql); //打印SQL
};
session.AttachOld(userList);
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + _rnd.Next(1, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
session.BeginTransaction();
foreach (SysUser user in userList)
{
session.Update(user);
}
session.CommitTransaction();
}
catch
{
session.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 测试循环添加
private void button6_Click(object sender, EventArgs e)
{
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("循环添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
var session = LiteSqlFactory.GetSession();
try
{
session.OnExecuting = (sql, param) =>
{
Console.WriteLine(sql); //打印SQL
};
session.BeginTransaction();
foreach (SysUser user in userList)
{
session.Insert(user);
}
session.CommitTransaction();
}
catch
{
session.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 查询
private void button1_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
var session = LiteSqlFactory.GetSession();
ISqlString sql = session.CreateSql(@"
select t.*
from sys_user t
where t.id > @id
and t.real_name like @remark", 20, "%测试%");
sql.Append(" order by t.create_time desc, t.id asc");
List<SysUser> userList = sql.QueryList<SysUser>();
Log("查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 分页查询
private void button2_Click(object sender, EventArgs e)
{
RunTask(() =>
{
Log("分页查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
int total = m_SysUserDal.GetTotalCount();
int pageSize = 100;
int pageCount = (total - 1) / pageSize + 1;
var session = LiteSqlFactory.GetSession();
List<SysUser> userList = new List<SysUser>();
for (int page = 1; page <= pageCount; page++)
{
ISqlString sql = session.CreateSql(@"
select t.*
from sys_user t
where 1=1
and t.id > @id
and t.real_name like @remark", 20, "%测试%");
string orderby = " order by t.create_time desc, t.id asc";
userList.AddRange(sql.QueryPage<SysUser>(orderby, pageSize, page));
}
Log("分页查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("分页查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发查询
private void button8_Click(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
RunTask(() =>
{
Log("并发查询 开始");
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
for (int i = 0; i < 200; i++)
{
Task task = RunTask(() =>
{
var session = LiteSqlFactory.GetSession();
ISqlString sql = session.CreateSql(@"
select t.*
from sys_user t
where t.id > @id
and t.real_name like @remark", 20, "%测试%");
sql.Append(" order by t.create_time desc, t.id asc");
List<SysUser> userList = sql.QueryList<SysUser>();
Log("查询结果 count=" + userList.Count.ToString());
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发插入
private void button9_Click(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
RunTask(() =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Remark = "测试插入用户" + i;
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("并发插入 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = RunTask(user =>
{
LiteSqlFactory.GetSession().Insert(user);
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发插入 完成,耗时:" + time + "秒");
});
}
#endregion
}
}
10. Dapper.LiteSql NET6版本
(为了和Fast.Framewor对比,用了异步,不过我用的MySql.Data.dll是假异步,Fast.Framework用的MySqlConnector是真异步):
using Dapper.LiteSql;
using Models;
using Provider;
namespace DapperLiteSqlDemo
{
public partial class Form1 : Form
{
#region 变量
private Random _rnd = new Random();
private int _count = 10000;
private bool _showSqlLog = false;
private static ILiteSqlClient _db;
#endregion
#region Form1
public Form1()
{
InitializeComponent();
}
#endregion
#region Form1_Load
private void Form1_Load(object sender, EventArgs e)
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
_db = new LiteSqlClient("server=localhost;database=litesql_test;user id=root;password=123456;character set=utf8mb4;",
DBType.MySQL, new MySQLProvider());
RunTask(() =>
{
_db.GetSession(); //预热
Log("预热完成");
});
}
#endregion
#region Log
private void Log(string log)
{
if (!this.IsDisposed)
{
string msg = DateTime.Now.ToString("mm:ss.fff") + " " + log + "\r\n\r\n";
if (this.InvokeRequired)
{
this.BeginInvoke(new Action(() =>
{
textBox1.AppendText(msg);
}));
}
else
{
textBox1.AppendText(msg);
}
}
}
#endregion
#region 清空输出框
private void button10_Click(object sender, EventArgs e)
{
textBox1.Text = string.Empty;
}
#endregion
#region RunTask
private Task RunTask(Action action)
{
return Task.Run(() =>
{
try
{
action();
}
catch (Exception ex)
{
Log(ex.ToString());
throw;
}
});
}
private Task RunTask<T>(Action<T> action, T t)
{
return Task.Factory.StartNew(obj =>
{
try
{
action((T)obj);
}
catch (Exception ex)
{
Log(ex.ToString());
throw;
}
}, t);
}
#endregion
#region 删除
private void button1_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
Log("删除 开始");
await _db.GetSession().CreateSql("id>@Id", 20).DeleteByConditionAsync<SysUser>();
Log("删除 完成");
});
}
#endregion
#region 批量修改
private void button2_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = await _db.GetSession().Queryable<SysUser>().Where(t => t.Id > 20).ToListAsync();
Log("批量修改 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
ISession? session = null;
try
{
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + _rnd.Next(1, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
session = await _db.GetSessionAsync();
session.BeginTransaction();
await session.UpdateAsync(userList);
session.CommitTransaction();
}
catch
{
session?.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 批量添加
private void button3_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("批量添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
ISession? session = null;
try
{
session = await _db.GetSessionAsync();
session.BeginTransaction();
await session.InsertAsync(userList);
session.CommitTransaction();
}
catch
{
session?.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("批量添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 循环修改
private void button4_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = await _db.GetSession().Queryable<SysUser>().Where(t => t.Id > 20).ToListAsync();
Log("循环修改 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
ISession? session = null;
try
{
foreach (SysUser user in userList)
{
user.Remark = "测试修改用户" + _rnd.Next(1, 10000);
user.UpdateUserid = "1";
user.UpdateTime = DateTime.Now;
}
session = await _db.GetSessionAsync();
session.BeginTransaction();
foreach (SysUser user in userList)
{
await session.UpdateAsync(user);
}
session.CommitTransaction();
}
catch
{
session?.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环修改 完成,耗时:" + time + "秒");
});
}
#endregion
#region 循环添加
private void button5_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("循环添加 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
ISession? session = null;
try
{
session = await _db.GetSessionAsync();
session.BeginTransaction();
foreach (SysUser user in userList)
{
await session.InsertAsync(user);
}
session.CommitTransaction();
}
catch
{
session?.RollbackTransaction();
throw;
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("循环添加 完成,耗时:" + time + "秒");
});
}
#endregion
#region 查询
private void button6_Click(object sender, EventArgs e)
{
RunTask(async () =>
{
Log("查询 开始");
DateTime dt = DateTime.Now;
for (int i = 0; i < 10; i++)
{
List<SysUser> userList = await _db.GetSession().Queryable<SysUser>()
.Where(t => t.Id > 20 && t.RealName.Contains("%测试%"))
.OrderByDescending(t => t.CreateTime)
.OrderBy(t => t.Id).ToListAsync();
Log("查询结果 count=" + userList.Count.ToString());
}
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 分页查询
private void button7_Click(object sender, EventArgs e)
{
Log("尚未实现");
}
#endregion
#region 并发查询
private void button8_Click(object sender, EventArgs e)
{
RunTask(() =>
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
Log("并发查询 开始");
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
for (int i = 0; i < 200; i++)
{
Task task = RunTask(async () =>
{
List<SysUser> userList = await _db.GetSession().Queryable<SysUser>()
.Where(t => t.Id > 20 && t.RealName.Contains("%测试%"))
.OrderByDescending(t => t.CreateTime)
.OrderBy(t => t.Id).ToListAsync();
Log("查询结果 count=" + userList.Count.ToString());
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发查询 完成,耗时:" + time + "秒");
});
}
#endregion
#region 并发插入
private void button9_Click(object sender, EventArgs e)
{
RunTask(() =>
{
ThreadPool.SetMaxThreads(1000, 1000);
ThreadPool.SetMinThreads(200, 200);
List<SysUser> userList = new List<SysUser>();
for (int i = 1; i <= _count; i++)
{
SysUser user = new SysUser();
user.UserName = "testUser";
user.RealName = "测试插入用户";
user.Remark = "测试插入用户" + i;
user.Password = "123456";
user.CreateUserid = "1";
user.CreateTime = DateTime.Now;
userList.Add(user);
}
Log("并发插入 开始 count=" + userList.Count);
DateTime dt = DateTime.Now;
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = RunTask(async user =>
{
await _db.GetSession().InsertAsync(user);
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
string time = DateTime.Now.Subtract(dt).TotalSeconds.ToString("0.000");
Log("并发插入 完成,耗时:" + time + "秒");
});
}
#endregion
}
}
部分测试结果
1. 循环插入、循环修改和查询
真的差不多。
2. 并发插入,全部通过
Fast.Framework还在优化,我用的临时版本
补充:Fast.Framework优化后正常了
(也就三两行代码的事,但不测试不知道):
SqlSugar和Fast.Framework的测试结果
就是我这篇博客要讲的的重点了,并发,肯定要加锁,并发大的时候,锁们可能在相互等待,干一会活,歇一会,导致白白浪费几秒,甚至几十秒。这种问题很难发现,看代码可能发现不了,因为从代码看可能代码写的没什么问题,你总不能不加锁吧?
真实原因是什么?线程池耗尽了
在这种情况下,.NET默认线程池1秒增加一个线程,刚增加的线程又被拿去跑任务了,然后MySqlConnection的Open方法就卡住了!是不是Open这个方法需要线程?而线程池中没有线程给它用了?
正常的是类似这样的,任务来了,就干活:
不正常的是类似这样的
(我写的Dapper.LiteSql,以前是使用using的没问题,后来为了干掉using,就加了连接池,就用到锁,锁的颗粒度大,就是像下面这样样子,比这还过份,后来把锁的粒度搞小一点,有改善,但还是不行,最后接着优化,除了用线程安全队列外(它底层也是锁),能不用锁就不用锁(有时候不用锁实现不了功能,又必须用),就解决了这个问题),下面是Fast.Framework优化前的测试:
耗时的全是外部调用:
3. 并发查询测试
我怀疑MySqlConnector的异步有BUG
Fast.Framework测试程序,明明用了Task.WaitAll(tasks.ToArray());,别人“并发查询 完成”都是最后输出,它还没完就输出了,这个3秒是不对的,实际是5秒,确实快,是不是有一部分是MySqlConnector真异步的功劳呢?
FreeSql报错(后续发现是数据库连接配置问题)
卡死了,报错。我之前大致看过它这个ObjectPool源码,没仔细研究,没太看懂,就觉得写的挺牛X,我还不太会。
补充:
上面是我坑了FreeSql,其它并发查询都是查200次,FreeSql是查了1000次,所以卡死,后改为查200次,不会卡死了,但是报错:
4. Winform确实会容易卡
主要是日志输出到界面导致的,减少输出日志的频率会好一些,查询1000次我刚试了一下Dapper.LiteSql,50多秒结束,其它就没测了。
结论
有问题很正常
我也只是测试了一种恶劣情况,说不定别人测试其它的恶劣情况,Dapper.LiteSql就报错了。
后续
这种写法是不是会好一点
try catch要加的,不过try catch可能也会有影响吧。下面是Dapper.LiteSql测试:
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = Task.Factory.StartNew(async user =>
{
try
{
await _db.GetSession().InsertAsync(user);
}
catch (Exception ex)
{
Log(ex.ToString());
}
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
Fast.Framework的测试
List<Task> tasks = new List<Task>();
foreach (SysUser item in userList)
{
var task = Task.Factory.StartNew(async user =>
{
try
{
await GetDbContext().Insert((SysUser)user).ExceuteAsync();
}
catch (Exception ex)
{
Log(ex.ToString());
}
}, item);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
题外话:
为什么.NET程序员喜欢写ORM?为什么.NET程序员ORM的话题经久不衰?
因为大家都自称增删改查程序员,增删改查做的多,但凡.NET老程序员,哪个没写过ADO.NET?ADO.NET的“脏活”都干的很多,非常熟悉了。
试想一下,写一写Redis的?ElasticSearch的?Kafka的?真没干过“脏活”,都是拿库来调用,简单的很,要写的话,就要去看文档,而且是英文文档,怎么写?其实大家并不是喜欢造轮子,只是对特别熟悉的东西造轮子,大家对ADO.NET就特别熟悉,增删改查的各种“脏活”没少写。
说到“脏活”,Socket的我写过一点,当时做一个.NET的WebApi,给Android端推消息(内网),没有用SuperSocket这种现成的,就自己写的Socket,项目专用,不通用,就没法做成开源库给大家用。因为应对的是特定需求,要写个通用的,就很费事了。我写个通用的给谁用?我自己可能都不愿意用。但ORM这东西,就算别人不用,我真还自己用!
出处:https://www.cnblogs.com/s0611163/p/16684103.html