换掉ES!Redis官方搜索引擎来了,性能炸裂!
2024/08/18 21:37:42
mall瀛︿範鏁欑▼瀹樼綉锛歮acrozheng.com
涔嬪墠灏变簡瑙h繃 RediSearch 杩欎釜椤圭洰锛屽苟鍦ㄩ」鐩腑浣跨敤杩囥€
灏辨垜鐨勪娇鐢ㄤ綋楠屾潵璇达紝绠€鍗曞満鏅笅锛岀敤鏉ュ钩鏇 Elasticsearch 鐨勪娇鐢ㄥ満鏅凡缁忚冻澶熴€傚儚鏄 Elasticsearch 涓父鐢ㄤ腑鏂囧垎璇嶆彃浠跺彲浠ョ敤 RediSearch 鏇夸唬锛屼絾鏄嫾闊宠浆涓枃鎻掍欢鍦 RediSearch 涓繕娌℃湁鍔熻兘鏇夸唬锛屽彧鑳介€氳繃涓汉鎵嬫澶勭悊銆
RediSearch 瀵逛簬鎴戞潵璇寸浉姣 Elasticsearch 鐨勬渶澶т紭鐐瑰氨鏄 鍐呭瓨鍗犵敤闈炲父浣庯紝鏌ヨ鎬ц兘涔熻冻澶熼珮馃槀銆
鍦ㄦ垜鐨勪綆閰 2 鏍 4g 鍐呭瓨鐨勬湇鍔″櫒涓婏紝閫氳繃瀹樻柟鎻愪緵鐨 Redis Stack 闀滃儚閮ㄧ讲 Redis 浠ュ強鑷甫妯″潡 RediSearch 鍚庯紝鍐呭瓨鍗犵敤鎵嶄笉鍒 100m銆
鐩告瘮閮ㄧ讲涓€涓 Elasticsearch 璧风爜闇€瑕 1g 鍐呭瓨鏉ヨ锛屾垜鏇存効鎰忛儴缃 RediSearch銆傛湰鏂囧ぇ绾插涓嬶紝
RediSearch 绠€浠
RediSearch 鏄竴涓 Redis 妯″潡锛屼负 Redis 鎻愪緵鏌ヨ銆佷簩绾х储寮曞拰鍏ㄦ枃鎼滅储鍔熻兘銆
瑕佷娇鐢 RediSearch 鐨勫姛鑳斤紝鎴戜滑闇€瑕佽鍏堝0鏄庝竴涓 index锛堢被浼间簬 Elasticsearch 鐨勭储寮曪級銆傜劧鍚庡氨鍙互浣跨敤 RediSearch 鐨勬煡璇㈣瑷€鏉ユ煡璇㈣绱㈠紩涓嬬殑鏁版嵁銆
RediSearch 鍐呴儴浣跨敤鍘嬬缉鐨勫€掓帓绱㈠紩锛屾墍浠ュ彲浠ョ敤杈冧綆鐨勫唴瀛樺崰鐢ㄦ潵瀹炵幇绱㈠紩鐨勫揩閫熸瀯寤恒€
鐩墠 RediSearch 鏈€鏂扮増鏀寔鐨勬煡璇㈠姛鑳戒篃姣旇緝涓板瘜浜嗭紝闄や簡鍩烘湰鐨勬枃鏈垎璇嶈繕鏀寔鑱氬悎缁熻銆佸仠鐢ㄨ瘝銆佸悓涔夎瘝銆佹嫾鍐欐鏌ャ€佺粨鏋滄帓搴忋€佹爣绛炬煡璇€佸悜閲忕浉浼煎害鏌ヨ浠ュ強涓枃鍒嗚瘝绛夈€
杩欐垨璁告槸涓€涓浣犳湁鐢ㄧ殑寮€婧愰」鐩紝mall椤圭洰鏄竴濂楀熀浜 SpringBoot + Vue + uni-app 瀹炵幇鐨勭數鍟嗙郴缁燂紙Github鏍囨槦60K锛夛紝閲囩敤Docker瀹瑰櫒鍖栭儴缃诧紝鍚庣鏀寔澶氭ā鍧楀拰寰湇鍔℃灦鏋勩€傚寘鎷墠鍙板晢鍩庨」鐩拰鍚庡彴绠$悊绯荤粺锛岃兘鏀寔瀹屾暣鐨勮鍗曟祦绋嬶紒娑电洊鍟嗗搧銆佽鍗曘€佽喘鐗╄溅銆佹潈闄愩€佷紭鎯犲埜銆佷細鍛樸€佹敮浠樼瓑鍔熻兘锛
- Boot椤圭洰锛歨ttps://github.com/macrozheng/mall
- Cloud椤圭洰锛歨ttps://github.com/macrozheng/mall-swarm
- 瑙嗛鏁欑▼锛歨ttps://www.macrozheng.com/video/
鍚庡彴绠$悊绯荤粺婕旂ず锛
鍓嶅彴鍟嗗煄椤圭洰婕旂ず锛
瀵规瘮 Elasticsearch
鍩烘湰纭欢
鏁版嵁婧
RediSearch 閰嶇疆
Elasticsearch 閰嶇疆
鐗堟湰
绱㈠紩鏋勫缓娴嬭瘯
鍦ㄥ畼鏂规彁渚涚殑绱㈠紩鏋勫缓娴嬭瘯涓紝RediSearch 鐢 221 绉掔殑閫熷害瓒呰繃浜 Elasticsearch 鐨 349 绉掞紝棰嗗厛 58%锛
鏌ヨ鎬ц兘娴嬭瘯
閫氳繃鏁版嵁闆嗗鍏ョ储寮曟暟鎹悗锛屽畼鏂逛娇鐢ㄨ繍琛屽湪涓撶敤璐熻浇鐢熸垚鍣ㄦ湇鍔″櫒涓婄殑 32 涓鎴风鍚姩浜嗕袱涓瘝鐨勬悳绱㈡煡璇€
濡備笅鍥炬墍绀猴紝RediSearch 鐨勫悶鍚愰噺杈惧埌浜 12.5K ops/sec锛岃€ Elasticsearch 鐨勫悶鍚愰噺鍙湁浜 3.1K ops/sec锛屽揩浜 4 鍊嶃€傛澶 RediSearch 鐨勫欢杩熺◢濂戒竴浜涳紝骞冲潎涓 8 姣锛岃€ Elasticsearch 涓 10 姣銆(ops/sec 姣忕鎿嶄綔鏁帮級
鐢辨鍙锛孯ediSearch 鍦ㄦ€ц兘涓婂姣 RediSearch 鏈夋瘮杈冨ぇ鐨勪紭鍔裤€
鐩墠 RediSearch 宸茬粡鏇存柊鍒 2.0+ 鐗堟湰锛屾牴鎹畼鏂瑰浜 RediSearch 2.0 鐗堟湰浠嬬粛锛屼笌 RediSearch 1.6 鐩告瘮锛屽悶鍚愰噺鍜屽欢杩熺浉鍏崇殑鎸囨爣閮芥彁楂樹簡 2.4 鍊嶃€
RediSearch 瀹夎
瀵逛簬鐩墠鏈€鏂扮殑 RediSearch 2.0 鐗堟湰鏉ヨ锛屽畼鏂规帹鑽愮洿鎺ヤ娇鐢 redis-stack-server 闀滃儚杩涜杩涜閮ㄧ讲锛屼篃姣旇緝绠€鍗曪紝
docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest
璁剧疆鐧诲綍瀵嗙爜
// 璁剧疆鐧诲綍瀵嗙爜
docker run -e REDIS_ARGS="--requirepass redis-stack" redis/redis-stack:latest
閫氳繃 redis-cli 杩炴帴鏌ョ湅 RediSearch 鏄惁瀹夎浜 search 妯″潡锛
redis-cli -h localhost
> MODULE list
...
3) 1) "name"
2) "search"
3) "ver"
4) "20809"
5) "path"
6) "/opt/redis-stack/lib/redisearch.so"
7) "args"
8) 1) "MAXSEARCHRESULTS"
2) "10000"
3) "MAXAGGREGATERESULTS"
4) "10000"
...
绱㈠紩鎿嶄綔
FT.CREATE 鍒涘缓绱㈠紩鍛戒护
> FT.CREATE idx:goods on hash prefix 1 "goods:" language chinese schema goodsName text sortable
"OK"
- FT.CREATE锛氬垱寤虹储寮曞懡浠
- idx:goods锛氱储寮曞悕绉
- on hash锛氱储寮曞叧鑱旂殑鏁版嵁绫诲瀷锛岃繖閲屾寚瀹氱储寮曞熀浜 hash 绫诲瀷鐨勬簮鏁版嵁鏋勫缓
- prefix 1 "goods:"锛氳〃绀虹储寮曞叧鑱旂殑 hash 绫诲瀷婧愭暟鎹墠缂€鏄 goods:
- language chinese锛氳〃绀烘敮鎸佷腑鏂囪瑷€鍒嗚瘝
- schema goodsName text sortable锛氳〃绀哄瓧娈靛畾涔夛紝goodsName 琛ㄧず鍏冩暟鎹睘鎬у悕锛宼ext 琛ㄧず瀛楁绫诲瀷 sortable 琛ㄧず璇ュ瓧娈靛彲浠ョ敤浜庢帓搴
娣诲姞绱㈠紩鏃讹紝鐩存帴浣跨敤 hset 鍛戒护娣诲姞涓€涓 key 鍓嶇紑鏄 "goods:" 鐨勬簮鏁版嵁銆傚涓嬶紝
hset goods:1001 goodsName 灏忕背鎵嬫満
hset goods:1002 goodsName 鍗庝负鎵嬫満
FT.SEARCH 鏌ヨ绱㈠紩
> FT.SEARCH idx:goods1 "鎵嬫満"
1) "2"
2) "goods:1001"
3) 1) "goodsName"
2) "\xe5\xb0\x8f\xe7\xb1\xb3\xe6\x89\x8b\xe6\x9c\xba"
4) "goods:1002"
5) 1) "goodsName"
2) "\xe5\x8d\x8e\xe4\xb8\xba\xe6\x89\x8b\xe6\x9c\xba"
FT.INFO 鏌ヨ鎸囧畾鍚嶇О绱㈠紩淇℃伅
> FT.INFO idx:goods
1) "index_name"
2) "idx:goods1"
3) "index_options"
4) (empty list or set)
5) "index_definition"
6) 1) "key_type"
2) "HASH"
3) "prefixes"
4) 1) "goods:"
5) "default_language"
6) "chinese"
7) "default_score"
8) "1"
7) "attributes"
8) 1) 1) "identifier"
2) "goodsName"
3) "attribute"
4) "goodsName"
5) "type"
6) "TEXT"
7) "WEIGHT"
8) "1"
9) "SORTABLE"
...
- FT.INFO 鏌ヨ鎸囧畾鍚嶇О鐨勭储寮曚俊鎭
FT.DROPINDEX 鍒犻櫎绱㈠紩鍚嶇О
> FT.DROPINDEX idx:goods1
"OK"
- FT.DROPINDEX 鍒犻櫎鎸囧畾鍚嶇О绱㈠紩锛屼笉浼氬垹闄 hash 绫诲瀷鐨勬簮鏁版嵁
濡傛灉闇€瑕佸垹闄ょ储寮曟暟鎹紝鐩存帴浣跨敤 del 鍛戒护鍒犻櫎绱㈠紩鍏宠仈鐨勬簮鏁版嵁鍗冲彲銆
Java 浣跨敤 RediSearch
瀵逛簬 Java 椤圭洰鐩存帴閫夌敤 Jedis4.0 浠ヤ笂鐗堟湰灏卞彲浠ヤ娇鐢 RediSearch 鎻愪緵鐨勬悳绱㈠姛鑳斤紝Jedis 鍦 4.0 浠ヤ笂鐗堟湰鑷姩鏀寔 RediSearch锛岀紪鍐 Jedis 杩炴帴 RedisSearch 娴嬭瘯鐢ㄤ緥锛岀敤 RedisSearch 鍛戒护鍒涘缓濡備笅锛
Jedis 鍒涘缓 RediSearch 瀹㈡埛绔
@Bean
public UnifiedJedis unifiedJedis(GenericObjectPoolConfig jedisPoolConfig) {
UnifiedJedis client;
if (StringUtils.isNotEmpty(password)) {
client = new JedisPooled(jedisPoolConfig, host, port, timeout, password, database);
} else {
client = new JedisPooled(jedisPoolConfig, host, port, timeout, null, database);
}
return client;
}
Jedis 鍒涘缓绱㈠紩
Schema schema = new Schema()
.addSortableTextField("goodsName", 1.0)
.addSortableTagField("tag", "|");
IndexDefinition rule = new IndexDefinition(IndexDefinition.Type.HASH)
.setPrefixes("idx:goods")
.setLanguage("chinese"); # 璁剧疆鏀寔涓枃鍒嗚瘝
client.ftCreate(idxName,
IndexOptions.defaultOptions().setDefinition(rule),
schema);
Jedis 娣诲姞绱㈠紩婧愭暟鎹
public boolean addGoodsIndex(String keyPrefix, Goods goods) {
Map<String, String> hash = MyBeanUtil.toMap(goods);
hash.put("_language", "chinese");
client.hset("idx:goods" + goods.getGoodsId(), MyBeanUtil.toMap(goods));
return true;
}
Jedis 涓枃鏌ヨ
public SearchResult search(String goodsIdxName, SearchObjVO searchObjVO, Page<SearchPageGoodsVO> page) {
// 鏌ヨ鍏抽敭瀛
String keyword = searchObjVO.getKeyword();
String queryKey = String.format("@goodsName:(%s)", keyword);
Query q = new Query(queryKey);
String sort = searchObjVO.getSidx();
String order = searchObjVO.getOrder();
// 鏌ヨ鏄惁鎺掑簭
if (StringUtils.isNotBlank(sort)) {
q.setSortBy(sort, Constants.SORT_ASC.equals(order));
}
// 璁剧疆涓枃鍒嗚瘝鏌ヨ
q.setLanguage("chinese");
// 璁剧疆鍒嗛〉
q.limit((int) page.offset(), (int) page.getSize());
// 杩斿洖鏌ヨ缁撴灉
return client.ftSearch(goodsIdxName, q);
}
鏈€鍚庤亰涓ゅ彞
RediSearch 鏄繖鍑犲勾鏂板嚭鐨勪竴涓叏鏂囨悳绱㈠紩鎿庯紝鍊熷姪浜 Redis 鐨勬垚鍔燂紝RediSearch 涓€鍑哄満灏辫幏寰椾簡杈冮珮鐨勫叧娉ㄥ害銆
鐩墠鏉ョ湅锛屾垜涓汉浣跨敤 RediSearch 浣滀负椤圭洰鐨勫叏鏂囨悳绱㈠紩鎿庡凡缁忓鐢ㄤ簡锛屽畠鏈夋槗浜庡畨瑁呫€佺储寮曞崰鐢ㄥ唴瀛樹綆銆佹煡璇㈤€熷害蹇瓑璁稿浼樼偣銆備笉杩囧湪瀵 Redis 闆嗙兢鐨勬敮鎸佷笂锛孯ediSearch 鐩墠鍙拡瀵 Redis 浼佷笟鐗堟湁瑙e喅鏂规锛屽紑婧愮増杩樻病鏈夛紝杩欎竴鐐归渶瑕佸憡璇夊ぇ瀹躲€
濡傛灉鎯宠鍦ㄧ敓浜х幆澧冨ぇ瑙勬ā浣跨敤锛屾垜杩樻槸涓嶅お寤鸿鐨勩€
Github涓
鏍囨槦60K鐨勭數鍟嗗疄鎴橀」鐩甿all锛屽叏濂 瑙嗛鏁欑▼锛2023鏈€鏂扮増锛 宸叉洿鏂板畬姣曪紒鍏ㄥ鏁欑▼绾40灏忔椂锛屽叡113鏈锛岄€氳繃杩欏鏁欑▼浣犲彲浠ユ嫢鏈変竴涓娑电洊涓绘祦Java鎶€鏈爤鐨勫畬鏁撮」鐩粡楠锛屽悓鏃舵彁楂樿嚜宸鐙珛寮€鍙戜竴涓」鐩殑鑳藉姏锛屼笅闈㈡槸椤圭洰鐨勬暣浣撴灦鏋勫浘锛屾劅鍏磋叮鐨勫皬浼欎即鍙互鐐瑰嚮閾炬帴 mall瑙嗛鏁欑▼ 鍔犲叆瀛︿範銆鏁村 瑙嗛鏁欑▼ 鐨勫唴瀹硅繕鏄潪甯稿畬鍠勭殑锛屾兜鐩栦簡mall椤圭洰鏈€浣冲涔犺矾绾裤€佹暣浣撴鏋舵惌寤恒€佷笟鍔′笌鎶€鏈疄鐜板叏鏂逛綅瑙f瀽銆佺嚎涓奃ocker鐜閮ㄧ讲銆佸井鏈嶅姟椤圭洰瀛︿範绛夊唴瀹癸紝鍏蜂綋澶х翰鍙互鍙傝€冧笅鍥撅紝浣犱篃鍙互鐐瑰嚮閾炬帴 mall瑙嗛鏁欑▼ 浜嗚В鏇村鍐呭銆
鎺ㄨ崘闃呰
- 69K Star锛佽繖鏄垜瑙佽繃鏈€寮虹殑寮€婧愮數鍟嗙郴缁 锛侊紒
- Github鏍囨槦60K锛佷竴濂楀畬鏁寸殑椤圭洰瀹炴垬鏁欑▼鏉ヤ簡锛屼富娴丣ava鎶€鏈竴缃戞墦灏斤紒
- 鐪嬩簡鎴戦」鐩腑璐墿杞︺€佽鍗曘€佹敮浠樹竴鏁村璁捐锛屽悓浜嬩篃寮€濮嬫倓鎮勬ā浠夸簡...
- 璁㈠崟绯荤粺灏辫杩欎箞璁捐锛岀ǔ鐨勪竴鎵癸紒
- 鏀粯绯荤粺灏辫杩欎箞璁捐锛岀ǔ鐨勪竴鎵癸紒
- 鏉冮檺绯荤粺灏辫杩欎箞璁捐锛岀ǔ鐨勪竴鎵癸紒