Commit Graph

282 Commits

Author SHA1 Message Date
phiresky
08401fc85f Revert "remove synchronous federation"
This reverts commit 2767ab4a6f.
2023-09-18 14:25:35 +00:00
phiresky
375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
Nutomic
71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
2023-09-06 10:56:26 -04:00
Nutomic
a0ea8dbc00
Auto resolve reports trigger (#3871)
* Revert "Automatically resolve report when post/comment is removed (#3850)"

This reverts commit f7f6766650.

* Automatically resolve reports using db trigger

* lint

* use mod log tables

* fix migration

* fix ci

* fix clippy
2023-09-05 05:35:10 -04:00
Dessalines
56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
2023-08-31 09:01:08 -04:00
Nutomic
7fd14b3d2a
Make remove content optional during account deletion (fixes #1617) (#3817)
* Make remove content optional during account deletion (fixes #1617)

* simplify purge params by passing context

* update js client

* add delete content

* update woodpecker config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-28 12:23:45 +02:00
Nutomic
6047257bfc
Move admin flag from person to local_user (fixes #3060) (#3403)
* Move admin flag from person to local_user (fixes #3060)

The person table is for federated data, but admin flag can only
apply to local users. Thats why it really belongs in the local_user
table. This will also prevent the federation code from accidentally
overwriting the admin flag

* fmt

* try to fix api tests

* lint

* fix person view

* ci

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-24 05:40:08 -04:00
Freakazoid182
c8063f3267
Replace Option<bool> with bool for PostQuery and CommentQuery (#3819) (#3857)
* Replace Option<bool> with bool for PostQuery and CommentQuery (#3819)

* Replace Option<bool> from all list queries (#3819)

---------

Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
2023-08-11 11:13:14 +02:00
Nutomic
f7f6766650
Automatically resolve report when post/comment is removed (#3850)
* Automatically resolve report when post/comment is removed

* also handle apub removes
2023-08-08 11:19:55 -04:00
dullbananas
969f8b2ce9
Replace TypedBuilder with Default in update forms (#3814)
* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update person.rs

* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update post.rs

* Update private_message.rs

* Update site.rs

* Update post.rs

* Update person.rs

* Update private_message.rs

* Update comment.rs

* Update create.rs

* Update leave_admin.rs

* Update update.rs

* Update remove.rs

* Update add_admin.rs

* Update verify_email.rs

* Update mod.rs

* Update mod.rs

* Update undo_delete.rs

* Update undo_delete.rs

* Update utils.rs

* Update feature.rs

* Update delete.rs

* Update lock.rs

* Update create.rs

* Update approve.rs

* Update update.rs

* Update lock_page.rs

* Update block_user.rs

* Update delete.rs

* Update undo_block_user.rs

* Update collection_remove.rs

* Update post.rs

* Update hide.rs

* Update person.rs

* Update remove.rs

* Update post_view.rs

* Update create.rs

* Update remove.rs

* Update collection_add.rs

* Update community.rs

* Update update.rs

* Update post_aggregates.rs

* Update update.rs

* Update comment.rs

* Update code_migrations.rs

* Update registration_application_view.rs

* Update update.rs

* Update ban_person.rs

* Update community.rs

* Update delete.rs

* Update delete.rs

* Update delete.rs

* Update person_aggregates.rs

* Update save_settings.rs

* Update distinguish.rs

* Update mark_read.rs

* Update site_aggregates.rs

* Update create.rs

* Fix

* rerun ci

* Update comment.rs

* rerun ci

* Update create.rs

* Update create.rs

* Update post_view.rs

* rerun ci

* Update undo_delete.rs

* rerun ci
2023-08-08 11:41:41 +02:00
Alex Lavallee
f9c2ba1cfa
Allow filtering PMs by sender (#3770)
* Allow filtering PMs by sender

* Fix retreiving own messages when filtering pm creator id

* Add tests

* Undo rename

* Fix PM tests

* Fix formatting for PMs

* Fix PM test formatting

* Rename PM creator_id field

* Fix PM formatting
2023-08-08 11:07:41 +02:00
Nutomic
2bb24c2859
Remove form_id params (#3812) 2023-08-04 09:22:43 -04:00
Dessalines
70fae9d68d
Fixing broken SQL migration formatting. (#3800)
* Fixing SQL format.

* Fixing clippy lints.

* Fixing clippy lint 2.
2023-08-03 11:07:35 +02:00
Nutomic
27be1efb74
Rewrite remaining federation actions, get rid of PerformCrud trait (#3794)
* Rewrite ban actions

* Rewrite delete/remove actions

* Rewrite remove/delete community

* Rewrite report actions

* Rewrite feature/lock post

* Rewrite update community actions

* Rewrite remaining federation actions

* Get rid of PerformCrud trait

* clippy
2023-08-02 12:52:41 -04:00
Nutomic
d82194cfe9
Rewrite some federation actions to remove Perform/SendActivity (ref #3670) (#3758) 2023-08-01 15:53:36 +02:00
Nutomic
37998b3398
Rewrite some API handlers to remove Perform trait (#3735)
* Rewrite some API handlers to remove Perform trait

* Convert CreateComment

* ci
2023-07-28 10:39:38 -04:00
Nutomic
3471f3533c
Sanitize html (#3708)
* HTML sanitization in apub code

* Sanitize API inputs

* fmt

* Dont allow html a, img tags

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-26 14:01:15 -04:00
Ben Wyatt
95e758e707
Bump version of dependency "webmention" (#3711)
Webmention .4->.5 bumps a few internal dependencies that needed to be upgraded
2023-07-25 13:01:13 -04:00
Nutomic
102124b6d2
Change logic for determining comment default language (fixes #3451) (#3672) 2023-07-21 09:08:25 -04:00
Nutomic
047db9ac85
Handle displaying of deleted and removed posts/comments (fixes #2624) (#3286)
* Handle displaying of deleted and removed posts/comments (fixes #2624)

* remove duplicate test

* fix tests

* no show_removed/show_deleted

* merge

* partially fix tests

* fix tests

* clippy

* fix tests

* get rid of build_post_response_deleted_allowed
2023-07-20 16:36:16 +02:00
Sander Saarend
d7051c40f8
Use local_site.default_post_listing_type as the initial default listing type for new users (#3666) 2023-07-19 11:04:43 -04:00
Nutomic
5d23ef960e
Remove SendActivity and Perform traits, rely on channel (#3596)
* Remove SendActivity and Perform traits, rely on channel

These traits arent necessary anymore now that websocket is removed.
Removing them allows us to use normal actix http handler methods
which are much more flexible, and allow using different middlewares
as well as setting response attributes.

* compiling and create post federating

* clippy

* rename methods, join outgoing activities task

* fix api tests

* no unwrap

* conditional compile

* add back getrandom

* make crates optional

* fmt
2023-07-19 09:49:41 -04:00
RocketDerp
38c6210912
Post remove delete federation outbound fix0 (#3613)
* add new function build_post_response_deleted_allowed

* PostDelete uses new function build_post_response_deleted_allowed

* RemovePost uses new build_post_response_deleted_allowed function

* code comments about mod or admin flag having other use

* reformat "cargo +nightly fmt --all"
2023-07-17 11:07:45 -04:00
cetra3
9256895635
Cache & Optimize Woodpecker CI (#3450)
* Try using drone cache plugin

* Try another path

* Include volume

* Fix formatting

* Include fmt

* Exclude cargo dir from prettier

* Don't override cargo

* Just do check

* Add cache key

* Use different cache plugin

* Add clippy

* Try minio

* Add quotes

* Try adding secrets

* Try again

* Again

* Use correct secret formation

* Add back clippy

* Use secret for the root bucket name

* Try drone cache instead

* Add region

* Add path-style option

* Include cargo clippy

* Include everything again

* Fix formatting

* Don't run clippy twice

* Add `allow` statements for tests to pass

* Adjust endpoint to be a secret

* Fix prettier

* Merge & fix tests

* Try to restart the woodpecker test

* Change the ENV var name

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-17 11:04:14 -04:00
dullbananas
88215bfbc9
Remove TypedBuilder from db_views and db_views_actor (#3637)
* change pool fields to parameters for list

* remove my_person_id and admin fields

* Change recipient id to list param

* Remove TypedBuilder from db_views and db_views_actor
2023-07-17 12:20:25 +02:00
Dessalines
164f4b93d9
Make lemmy_api_common wasm-compatible (#3587)
* Fixing lemmy_api_common optionals

* Adding taplo for .toml files.

* Versioning taplo in woodpecker.

* Addressing PR comments.
2023-07-14 10:45:18 +02:00
dullbananas
1d38aad9d3
Make functions work with both connection and pool (#3420)
* a lot

* merge

* Fix stuff broken by merge

* Get rid of repetitive `&mut *context.conn().await?`

* Add blank lines under each line with `conn =`

* Fix style mistakes (partial)

* Revert "Fix style mistakes (partial)"

This reverts commit 48a033b87f.

* Revert "Add blank lines under each line with `conn =`"

This reverts commit 773a6d3beb.

* Revert "Get rid of repetitive `&mut *context.conn().await?`"

This reverts commit d2c6263ea1.

* Use DbConn for CaptchaAnswer methods

* DbConn trait

* Remove more `&mut *`

* Fix stuff

* Re-run CI

* try to make ci start

* fix

* fix

* Fix api_common::utils

* Fix apub::activities::block

* Fix apub::api::resolve_object

* Fix some things

* Revert "Fix some things"

This reverts commit 2bf8574bc8.

* Revert "Fix apub::api::resolve_object"

This reverts commit 3e4059aabb.

* Revert "Fix apub::activities::block"

This reverts commit 3b02389abd.

* Revert "Fix api_common::utils"

This reverts commit 7dc73de613.

* Revert "Revert "Fix api_common::utils""

This reverts commit f740f115e5.

* Revert "Revert "Fix apub::activities::block""

This reverts commit 2ee206af7c.

* Revert "Revert "Fix apub::api::resolve_object""

This reverts commit 96ed8bf2e9.

* Fix fetch_local_site_data

* Fix get_comment_parent_creator

* Remove unused perma deleted text

* Fix routes::feeds

* Fix lib.rs

* Update lib.rs

* rerun ci

* Attempt to create custom GetConn and RunQueryDsl traits

* Start over

* Add GetConn trait

* aaaa

* Revert "aaaa"

This reverts commit acc9ca1aed.

* Revert "Revert "aaaa""

This reverts commit 443a2a00a5.

* still aaaaaaaaaaaaa

* Return to earlier thing

Revert "Add GetConn trait"

This reverts commit ab4e94aea5.

* Try to use DbPool enum

* Revert "Try to use DbPool enum"

This reverts commit e4d1712646.

* DbConn and DbPool enums (db_schema only fails to compile for tests)

* fmt

* Make functions take `&mut DbPool<'_>` and make db_schema tests compile

* Add try_join_with_pool macro and run fix-clippy on more crates

* Fix some errors

* I did it

* Remove function variants that take connection

* rerun ci

* rerun ci

* rerun ci
2023-07-11 09:09:59 -04:00
Pawan Hegde
ef9dc5d0b6
Fix #3366: Wrap plain-text error responses from the API in JSON (#3559)
* Fix #3366: API does return plain HTML errors

* Fix Clippy errors

* Improve api response times by doing send_activity asynchronously (#3493)

* do send_activity after http response

* move to util function

* format

* fix prometheus

* make synchronous federation configurable

* cargo fmt

* empty

* empty

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>

* Updating `login.rs` with generic `incorrect_login` response. (#3549)

* Adding v0.18.1 and v0.18.0 release notes. (#3530)

* Update RELEASES.md (#3556)

added instruction to find the location of your docker directory (especially useful for those who used ansible since they never had to setup docker manually)

* Use async email sender (#3554)

* Upgrade all dependencies (#3526)

* Upgrade all dependencies

* as base64

* Adding phiresky to codeowners. (#3576)

* Error enum fixed (#3487)

* Create error type enum

* Replace magic string slices with LemmyErrorTypes

* Remove unused enum

* Add rename snake case to error enum

* Rename functions

* clippy

* Fix merge errors

* Serialize in PascalCase instead of snake_case

* Revert src/lib

* Add serialization tests

* Update translations

* Fix compilation error in test

* Fix another compilation error

* Add code for generating typescript types

* Various fixes to avoid breaking api

* impl From<LemmyErrorType> for LemmyError

* with_lemmy_type

* trigger ci

---------

Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>

* Only update site_aggregates for local site (#3516)

* Fix #3501 - Fix aggregation counts for elements removed and deleted (#3543)

Two bugs were found and fixed:
- previously elements removal and deletion were counted as two separate disappearances
- removing comments did not affect post aggregations

* Use LemmyErrorType also make error_type compulsory

* Add missing import for jsonify_plain_text_errors

* Fix formatting

* Trying to make woodpecker run again

---------

Co-authored-by: phiresky <phireskyde+git@gmail.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: rosenjcb <rosenjcb@gmail.com>
Co-authored-by: nixoye <12674582+nixoye@users.noreply.github.com>
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
Co-authored-by: Sander Saarend <sander@saarend.com>
Co-authored-by: Piotr Juszczyk <74842304+pijuszczyk@users.noreply.github.com>
2023-07-10 22:44:14 +02:00
Nutomic
93225e5ddf
Error enum fixed (#3487)
* Create error type enum

* Replace magic string slices with LemmyErrorTypes

* Remove unused enum

* Add rename snake case to error enum

* Rename functions

* clippy

* Fix merge errors

* Serialize in PascalCase instead of snake_case

* Revert src/lib

* Add serialization tests

* Update translations

* Fix compilation error in test

* Fix another compilation error

* Add code for generating typescript types

* Various fixes to avoid breaking api

* impl From<LemmyErrorType> for LemmyError

* with_lemmy_type

* trigger ci

---------

Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
2023-07-10 16:50:07 +02:00
dullbananas
934f72511e
Use async email sender (#3554) 2023-07-10 14:04:39 +02:00
phiresky
b35757b429
Improve api response times by doing send_activity asynchronously (#3493)
* do send_activity after http response

* move to util function

* format

* fix prometheus

* make synchronous federation configurable

* cargo fmt

* empty

* empty

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-10 12:27:49 +02:00
Nutomic
00f9f79a44
Only allow http(s) scheme for urls (ref #3505) (#3508)
With this change only http(s) schemes are allowed for post.url
field. This is checked for incoming api and federation requests.
Existing posts in database which are sent to clients are not
checked. Neither does it check urls in markdown.
2023-07-06 08:29:51 -04:00
Nutomic
e1494d4683
Dont compare db string errors (fixes #1393) (#3424)
* Dont compare db string errors (fixes #1393)

* cargo fmt

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-03 11:59:49 -04:00
Lemmus.org
810762762f
Update federated posts to not cache sensitive images if not allow by local site (#3253)
* Update federated posts to not cache sensitive images if not allow by local site

* Refactor thumbnail match to simplify logic
2023-06-30 12:42:42 +02:00
Sander Saarend
ec18fd9869
Fix cargo warnings (#3397) 2023-06-28 12:57:49 -04:00
dullbananas
bef76630c5
Remove redundant calls to Iterator::collect (#3365)
* Remove redundant calls to `Iterator::collect`

* Update mentions.rs

* Add clippy lints and run fmt

* CI ran on the wrong commit again 
2023-06-28 11:19:26 +02:00
Nina Blanson
e63aa80c3a
Fixes #2900 - Checks slur regex to see if it is too permissive (#3146)
* Fixes #2900 - Checks slur regex to see if it is too permissive along with small validation organization

* Clean up variable names, add handler for valid empty string usecase

* Update tests

* Create validation function and add tests

* Test clean up

* Use payload value vs local site value to prevent stunlocking

* Remove println added while testing

* Fall back to local site regex if not provided from request

* Attempt clean up of flaky comment_view tests

* Pull in latest submodule

* Move application, post check into functions, add more tests and improve test readability

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-06-27 07:03:30 -04:00
Domenic Horner
d1d90af0eb
add new flag to api (#3363) 2023-06-27 06:45:26 -04:00
TKilFree
2aef6a5a33
feat: re-added captcha checks (#3289) 2023-06-27 06:38:53 -04:00
Neshura
203e35899e
Add separate Post check for is_valid_body_field (#3263)
* Add separate Post check for is_valid_body_field

* Modify is_valid_body_check for posts only

* Fix check var reinit in validation.rs

* Extra empty line to rerun woodpecker with changes

* Change Option to bool, add false to non-post calls

* Woodpecker trick.. again

* Probable rust_fmt fail fixed

* cargo_clippy changes

* Missing space between = and if

* Remove ; after body length checks
2023-06-26 10:47:01 +02:00
cetra3
d7da911a48
Remove actix_rt & use standard tokio spawn (#3158)
* Remove `actix_rt` & use standard tokio spawn

* Adjust rust log back down

* Format correctly

* Update cargo lock

* Add DB settings

* Change name and update to latest rev

* Clean up formatting changes

* Move `worker_count` and `worker_retry_count` to settings

* Update defaults

* Use `0.4.4` instead of git branch
2023-06-26 10:24:11 +02:00
Dessalines
1f1c0ad17a
Revert "feat: re-added captcha checks (#3249)" (#3288)
This reverts commit 8a086c8240.
2023-06-23 07:02:05 -04:00
TKilFree
8a086c8240
feat: re-added captcha checks (#3249) 2023-06-23 11:47:12 +02:00
Scott
da3e3b6d8a
Ensure site can only either be a private instance or federated when creating or editing site (#3237)
* Add site visibility validation

* Fix formatting

* linter changes

* Update error message to match existing check

* Remove existing check

---------

Co-authored-by: Scott Merchant <97430840+scottmerchant@users.noreply.github.com>
2023-06-21 22:14:12 +02:00
Nutomic
e3eda68147
Correct error messages if user registers with taken user/email (#3093)
* Correct error messages if user registers with taken user/email (fixes #2955)

* exists
2023-06-21 05:26:07 -04:00
Dessalines
1ef095dd96
Fixing cross_post duplicates. Fixes #3233 (#3234) 2023-06-21 10:56:44 +02:00
Wyatt Smith
744153eee6
Prevent making an instance private if federation is enabled. (#3074)
* Prevent making an instance private if federation is enabled.

* Added case where federation is enabled, check if the alternative setting is changed

* Merged the error messages for private instance/federation.

---------

Co-authored-by: Wyatt Smith <wys@dropbox.com>
2023-06-15 11:35:09 +02:00
Dessalines
5af831c6fa
Fix unapproved users being able to log in after registration mode set to closed. (#3096)
- Fixes #3095
2023-06-14 19:02:17 -04:00
Andy Brandt
f98476c5c0
Fix private message e-mail notification subject and body (#2963)
It used to say:

Subject: Private message from [your username]
Body: [your username] - [private message]

It now display the correct username that it's from.
2023-06-08 13:42:32 -04:00
Anon
15c84e2f7b
Add comment depth check (#2940)
* Add comment depth check

* Move comment depth code

* linter fix
2023-06-08 10:31:26 -04:00