Commit graph

324 commits

Author SHA1 Message Date
Nathan.fooo
3e65dd1014
add docs and tests for embedding related (#1328)
* chore: add docs and tests

* chore: add test

* chore: fix typo

* chore: clippy
2025-04-10 23:12:53 +08:00
Nathan.fooo
3181b17d60
chore: set deployment id for azure embedding (#1322)
* chore: fix audit

* chore: update audit config

* chore: fix azure embedding

* chore: adjust ai config

* fix: do not generate embedding when all chunk content is empty
2025-04-07 14:44:22 +08:00
Nathan.fooo
2c1c820e68
chore: support azure open ai (#1321) 2025-04-06 19:11:40 +08:00
Bartosz Sypytkowski
1fd900d994
Document paragraphs (#1119)
* chore: create embeddings by paragraphs

* chore: use document paragraphs method

* chore: document indexing by paragraphs with consistent hash

* chore: compare produced embeddings against existing ones

* chore: make pg stored proc compare between input and existing embedded fragments

* chore: missing sqlx generation

* fix: appflowy worker

* chore: make sure that embeddings are only changed when content had changed

* chore: remove partition key and recreate af_collab_embeddings_upsert migration

* chore: use pg15 on CI and update af_collab_embeddings table primary key

* chore: fix test

---------

Co-authored-by: Nathan <nathan@appflowy.io>
2025-04-06 17:47:02 +08:00
Bartosz Sypytkowski
ac834adc2f
chore: add information about object id to log errors in apply update (#1315) 2025-04-03 14:59:24 +08:00
Bartosz Sypytkowski
eaee9d3317 chore: cargo clippy fix 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
576f1654cf chore: post cargo rebase fixes 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
f2d5625068 chore: cargo fmt 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
a591f7bd96 chore: post merge fixes 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
ec45ef23fe chore: first phase of tests 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
2687127310 chore: cargo clippy fix 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
1db7cb36e0 chore: cargo clippy fix 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
b2aa5c39c1 chore: fix clippy errors 2025-04-01 08:19:43 +02:00
Bartosz Sypytkowski
0cedc39856 chore: ported uuids for everything except tests 2025-04-01 08:18:12 +02:00
Bartosz Sypytkowski
a5d6cc0a59 chore: passed through first circle of hell 2025-04-01 07:42:13 +02:00
khorshuheng
92495aa404 chore: init env filter by parsing the default env variable 2025-03-25 17:04:58 +08:00
Khor Shu Heng
a808b7832b
chore: use uuid for workspace access control (#1293) 2025-03-25 13:09:41 +08:00
Khor Shu Heng
f14ce3d9d4
chore: remove unused code related to experimental standalone collab service (#1295) 2025-03-25 13:09:24 +08:00
khorshuheng
9c435d9016 feat: add favorite reorder API 2025-03-24 12:55:22 +08:00
Bartosz Sypytkowski
ac1ebf6646 chore: awareness pubsub switch to lightweight version 2025-03-19 09:04:07 +01:00
マルコメ
59b284ed6d
chore: add syntax parser directive to Dockerfile (#1271) 2025-03-16 10:28:32 +08:00
Bartosz Sypytkowski
30d6b4edb2 chore: change awareness broadcasting from stream to pubsub 2025-03-11 05:25:22 +01:00
khorshuheng
d9d511be99 chore: remove unnecessary package as reported by cargo machete 2025-02-17 16:26:01 +08:00
Khor Shu Heng
b8d8007f26
fix: issue:991 unable to import from Notion when using minio (#1203) 2025-02-05 09:55:56 +08:00
Nathan.fooo
b7e08687b4
chore: use curl for gotrue health check when using docker-compose-ci.yml (#1189) 2025-01-22 21:40:47 +08:00
Khor Shu Heng
e958dda0b8
chore: remove history modules (#1174) 2025-01-19 22:45:49 +08:00
Khor Shu Heng
c3bba778a2
Merge pull request #1163 from AppFlowy-IO/trim-awareness-on-drop
fix: trim awareness when collab group is dropped
2025-01-15 11:33:44 +08:00
khorshuheng
8c6a657f03 fix: trim awareness when collab group is dropped 2025-01-15 00:54:49 +08:00
Nathan.fooo
56f4c0698c
chore: update subscription entity (#1162) 2025-01-14 17:35:27 +08:00
Nathan.fooo
c5cdb06b77
chore: notify user file size (#1161) 2025-01-14 12:50:52 +08:00
Nathan.fooo
5a2f54bfdd
chore: delete imported task stream value (#1160)
* chore: delete import task

* chore: delete requeue task
2025-01-14 12:01:52 +08:00
Nathan.fooo
3211a93caa
chore: update expire time for database row (#1158) 2025-01-14 10:15:34 +08:00
Khor Shu Heng
fa9d53461b
feat: add web guide as part of getting started guide (#1142) 2025-01-10 16:30:05 +08:00
Nathan.fooo
b47a635cfc
chore: support async embed (#1139) 2025-01-08 16:37:17 +08:00
Nathan.fooo
e0bc8f814b
chore: update env (#1136) 2025-01-07 15:47:36 +08:00
Nathan.fooo
5195dae3a5
feat: Chat file metadata (#1133)
* chore: chat question metadata

* chore: update

* chore: update env

* chore: update env
2025-01-07 12:46:55 +08:00
Bartosz Sypytkowski
cc6b58f73d chore: add metrics to redis collab stream 2025-01-03 08:44:36 +01:00
Nathan.fooo
3ee52ad8d0
chore: fix private open api key in config (#1112)
* chore: fix private open api key in config

* chore: fmt
2025-01-02 13:03:01 +08:00
Bartosz Sypytkowski
a0bed04685 chore: add prometheus histogram metrics for collab redis update read latency 2024-12-31 14:18:42 +01:00
Nathan.fooo
b886db1d5c
chore: fix clippy warnings (#1103) 2024-12-29 12:47:34 +08:00
Bartosz Sypytkowski
4550cae2b1 chore: cherry pick github pr 1054 2024-12-27 10:07:01 +01:00
Bartosz Sypytkowski
c94938ee46 chore: fix clippy errors 2024-12-27 07:52:10 +01:00
khorshuheng
824d60920d Merge branch 'main' into stateless 2024-12-26 11:55:52 +08:00
Nathan.fooo
c80491267b
chore: update metrics and num threads (#1102) 2024-12-26 11:55:54 +08:00
nathan
fddbc8073e chore: fix test 2024-12-24 15:42:52 +08:00
nathan
1a5c1714db Merge branch 'main' into stateless 2024-12-24 14:59:04 +08:00
Nathan.fooo
1131818eb7
fix: support index document in the background, fix stack overflow when calling rayon::spawn (#1099)
* chore: batch index

* chore: format log

* chore: index workspace

* chore: fix stack overflow

* chore: background index

* chore: clippy

* chore: filter tasks

* chore: clippy

* chore: add metrics

* chore: fix test
2024-12-24 14:30:17 +08:00
Bartosz Sypytkowski
837f4de8f3 chore: remove gc check 2024-12-23 14:57:52 +01:00
Bartosz Sypytkowski
35430c6be2 chore: remove doing snapshots in collab group periodic saves 2024-12-23 13:02:23 +01:00
Bartosz Sypytkowski
59e8281010 Merge branch 'main' into stateless 2024-12-23 05:21:15 +01:00