Commit graph

192 commits

Author SHA1 Message Date
khorshuheng
553d7101bd feat: api to get invitation code info 2025-04-16 10:26:54 +08:00
Nathan.fooo
3e65dd1014
add docs and tests for embedding related (#1328)
* chore: add docs and tests

* chore: add test

* chore: fix typo

* chore: clippy
2025-04-10 23:12:53 +08:00
Bartosz Sypytkowski
1fd900d994
Document paragraphs (#1119)
* chore: create embeddings by paragraphs

* chore: use document paragraphs method

* chore: document indexing by paragraphs with consistent hash

* chore: compare produced embeddings against existing ones

* chore: make pg stored proc compare between input and existing embedded fragments

* chore: missing sqlx generation

* fix: appflowy worker

* chore: make sure that embeddings are only changed when content had changed

* chore: remove partition key and recreate af_collab_embeddings_upsert migration

* chore: use pg15 on CI and update af_collab_embeddings table primary key

* chore: fix test

---------

Co-authored-by: Nathan <nathan@appflowy.io>
2025-04-06 17:47:02 +08:00
khorshuheng
dba8db995d feat: join workspace by invite code 2025-04-03 22:12:38 +08:00
Bartosz Sypytkowski
a7d078414c chore: cargo sqlx fix 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
ec45ef23fe chore: first phase of tests 2025-04-01 08:22:36 +02:00
Bartosz Sypytkowski
c2e14a9639 chore: working on query conversions 2025-04-01 07:42:13 +02:00
Bartosz Sypytkowski
6e6e25fb14 chore: mid-way 2025-04-01 07:42:13 +02:00
Nathan.fooo
06ddd7f755
chore: generate ai image if need (#1254)
* chore: generate ai image

* chore: fix test
2025-02-26 22:54:03 +08:00
khorshuheng
672118355e feat: fetch workspace member via uuid 2025-02-17 11:34:32 +08:00
khorshuheng
fb15b79909 chore: remove unused collab member crud api 2025-02-13 10:48:19 +08:00
khorshuheng
44dee16e64 feat: add comments enabled and duplicate enabled to publish info 2025-01-20 14:30:14 +08:00
Khor Shu Heng
0f7a1f41d0
feat: support disabling comments and disallow duplicate as template for published page (#1167) 2025-01-17 09:43:56 +08:00
Khor Shu Heng
a5d94a09d6
feat: add publish info to publish outline items (#1156) 2025-01-12 13:08:38 +08:00
Nathan.fooo
ee84464e82
chore: store blob file status (#1147)
* chore: store blob file status

* chore: fmt

* chore: fmt
2025-01-10 14:52:35 +08:00
khorshuheng
6d09fbb236 fix: select most recent non original workspace 2025-01-08 21:24:56 +08:00
khorshuheng
ace49ef18f feat: endpoint for unpublish page 2025-01-07 12:25:48 +08:00
nathan
1a5c1714db Merge branch 'main' into stateless 2024-12-24 14:59:04 +08:00
Nathan.fooo
1131818eb7
fix: support index document in the background, fix stack overflow when calling rayon::spawn (#1099)
* chore: batch index

* chore: format log

* chore: index workspace

* chore: fix stack overflow

* chore: background index

* chore: clippy

* chore: filter tasks

* chore: clippy

* chore: add metrics

* chore: fix test
2024-12-24 14:30:17 +08:00
Bartosz Sypytkowski
35430c6be2 chore: remove doing snapshots in collab group periodic saves 2024-12-23 13:02:23 +01:00
Bartosz Sypytkowski
59e8281010 Merge branch 'main' into stateless 2024-12-23 05:21:15 +01:00
Nathan.fooo
46f9c7811f
chore: return unindex collab one by one (#1095)
* chore: return unindex collab one by one

* chore: clippy
2024-12-20 15:14:47 +08:00
khorshuheng
8674c9f3ad Merge branch 'main' into stateless 2024-12-20 13:24:44 +08:00
khorshuheng
414d721ee0 feat: quick note CRUD APIs 2024-12-20 11:11:51 +08:00
khorshuheng
1b17c8586b Merge branch 'quick-notes-api' into stateless 2024-12-19 16:50:33 +08:00
khorshuheng
8128c2f8e5 feat: quick note CRUD APIs 2024-12-19 16:50:05 +08:00
Bartosz Sypytkowski
9d6b272656 chore: merge main into stateless branch 2024-12-19 06:52:20 +01:00
Richard Shiue
ecadf8e287
chore: find question message from reply message (#1085)
* chore: find question message from answer message id

* chore: sqlx

* test: fix tests

* test: fix test

* chore: apply code suggestions to 2 files
2024-12-19 00:12:53 +08:00
Bartosz Sypytkowski
d5252f4273 Merge branch 'main' into stateless 2024-12-17 18:24:17 +01:00
Nathan.fooo
abf827f2a6
chore: query multiple collab embedding state (#1081)
* chore: query multiple collab embedding state

* chore: clippy
2024-12-17 23:23:43 +08:00
Bartosz Sypytkowski
1b35326f0a Merge branch 'main' into stateless 2024-12-16 04:47:26 +01:00
Nathan.fooo
2ab1e94b99
chore: create embed in background (#1071)
* refactor: create indexer scheduler

* refactor: replace current calls

* chore: use rayon thread pool

* chore: index unindexed collab using thread pool

* chore: index collab when write to disk

* chore: add test

* chore: add test

* chore: add test

* chore: fix client api deps

* chore: chat with document test

* chore: fix test

* chore: clippy

* chore: update test
2024-12-15 12:29:51 +08:00
khorshuheng
c1c5bf7420 chore: stop writing to af_collab_member 2024-12-13 10:17:22 +08:00
Bartosz Sypytkowski
41e9302ff9 Merge branch 'main' into stateless 2024-12-12 16:00:45 +01:00
Nathan.fooo
af38efe6d9
feat: sync document through http request (#1064)
* chore: query embedding

* chore: create embeddings

* chore: apply update to editing collab

* refactor: web-update

* chore: calculate missing update the sv is not none

* chore: add test

* chore: fix audit

* chore: commit sqlx

* chore: fix client api

* test: add

* chore: clippy

* chore: fix collab drop when save
2024-12-12 14:53:07 +08:00
khorshuheng
a8c3adf166 feat: shift admin user creation responsibility to gotrue service 2024-12-08 15:01:48 +08:00
Bartosz Sypytkowski
66deaf3fa3
chore: move collab cache for appflowy collaborate service and add metrics (#1045)
* chore: move collab cache for appflowy collaborate service and add metrics

* chore: add histogram for measuring time spent in postgres transactions

* chore: delete blobs in chunks

* chore: bring back redis cache filling by import worker

* chore: cargo fmt

* chore: make import workspace update existing collabs

* chore: bulk insert collab deduplicate records
2024-12-06 19:55:51 +08:00
Nathan.fooo
afcd1130c3
feat: chat settings (#1044)
* feat: chat settings

* chore: fix sqlx
2024-12-05 23:30:11 +08:00
Bartosz Sypytkowski
5e7b5f87fa chore: merge branch main into stateless 2024-12-03 09:10:01 +01:00
Bartosz Sypytkowski
9ff6f1c744
S3 collab cache (#1028)
* chore: collab cache for S3

* chore: adjust disk cache api to accomodate s3

* chore: move postgres dependent ops to disk cache

* chore: replace blob inserts from pg to s3

* chore: delete blob and collab exist now use s3

* chore: fix clippy erorrs

* chore: post rebase fixes

* chore: fix clippy warnings

* chore: fix imports

* chore: make snapshots work over S3

* chore: remove dead code

* chore: use compressed snapshots

* chore: add zstd compression

* chore: introduce collab size threshold to keep smaller collabs in postgres

* chore: remove collabs from S3 if they were put to postgres

* chore: update tests
2024-12-03 13:08:55 +08:00
Bartosz Sypytkowski
4fb6df5c14
chore: reduce number of SQL statements executed when inserting/updating collab (#1022)
* chore: reduce number of SQL statements executed when inserting new collab

* chore: add workspace id check

* chore: fix previous query

* chore: upsert collabs and embddings as stored procedures

* chore: fix clippy errors

* chore: fix stored procedure to compare workspaces correctly
2024-11-27 13:00:00 +08:00
Zack Fu Zi Xiang
c4d52859fb
feat: add a polling api for getting database row id updates 2024-11-24 23:24:29 +08:00
Bartosz Sypytkowski
1932607ca9 Merge branch 'main' into stateless 2024-11-18 05:43:16 +01:00
Nathan.fooo
655f13bc27
chore: search and chat (#999)
* chore: add test for search and chat

* chore: update test

* chore: update test

* chore: update ci

* chore: fix security audio

* chore: multiple core docker build

* chore: multiple core docker build

* chore: update ci

* chore: update model setting

* chore: test ci

* chore: use tiktoken to calcualte token length

* chore: remove env

* chore: use spawn_blocking with condition

* chore: docs

* chore: clippy

* chore: clippy

* chore: docker logs

* chore: pass message id

* chore: clippy
2024-11-16 14:52:12 +08:00
Bartosz Sypytkowski
401cc3ad25 Merge branch 'main' into stateless 2024-11-16 07:22:34 +01:00
Bartosz Sypytkowski
f6fef9918b
chore: split document content into multiple embeddings (#990)
* chore: split document content into multiple embeddings

* chore: fallback to uuid v4 generation due to unmaintained transient dependency

* chore: fix lint errors
2024-11-14 14:15:52 +08:00
Zack Fu Zi Xiang
f3cfd87cb8
fix: patch publish name 2024-11-13 17:21:49 +08:00
Zack
85452ddfab
feat: add role for user when querying workspaces (#983) 2024-11-12 21:12:58 +08:00
Zack Fu Zi Xiang
ed4c14d53a
fix: also specify workspace id when deleting collab 2024-11-11 17:31:44 +08:00
Zack Fu Zi Xiang
b62552f382
fix: fetch published collab sql 2024-11-11 00:24:30 +08:00