mediawiki/services/cxserver: master (log #252381)

sourcepatches
$ date
Wed Apr 28 11:06:50 UTC 2021

$ git clone file:///srv/git/mediawiki-services-cxserver.git repo --depth=1 -b master
Cloning into 'repo'...

$ git config user.name libraryupgrader

$ git config user.email tools.libraryupgrader@tools.wmflabs.org

$ git submodule update --init

$ grr init
Installed commit-msg hook.

$ git show-ref refs/heads/master
b4b147ae3290b6a1c78df7a6363a447399695d3a refs/heads/master

Attempting to npm audit fix
$ npm audit fix --only=dev

> nodemon@2.0.7 postinstall /src/repo/node_modules/nodemon
> node bin/postinstall || exit 0

Love nodemon? You can now support the project via the open collective:
 > https://opencollective.com/nodemon/donate

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.3.2 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 544 packages from 254 contributors in 8.962s

52 packages are looking for funding
  run `npm fund` for details

fixed 0 of 4 vulnerabilities in 931 scanned packages

Verifying that tests still pass
$ npm ci
npm WARN prepare removing existing node_modules/ before installation

> sqlite3@4.2.0 install /src/repo/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
[sqlite3] Success: "/src/repo/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" is installed via remote

> dtrace-provider@0.8.8 install /src/repo/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js

make: Entering directory '/src/repo/node_modules/dtrace-provider/build'
  TOUCH Release/obj.target/DTraceProviderStub.stamp
make: Leaving directory '/src/repo/node_modules/dtrace-provider/build'

> unix-dgram@2.0.4 install /src/repo/node_modules/unix-dgram
> node-gyp rebuild

make: Entering directory '/src/repo/node_modules/unix-dgram/build'
  CXX(target) Release/obj.target/unix_dgram/src/unix_dgram.o
In file included from ../src/unix_dgram.cc:5:
../../nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../../nan/nan.h:2294:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from ../../nan/nan.h:56,
                 from ../src/unix_dgram.cc:5:
../src/unix_dgram.cc: At global scope:
/cache/node-gyp/10.24.0/include/node/node.h:573:43: warning: cast between incompatible function types from ‘void (*)(v8::Local<v8::Object>)’ to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
       (node::addon_register_func) (regfunc),                          \
                                           ^
/cache/node-gyp/10.24.0/include/node/node.h:607:3: note: in expansion of macro ‘NODE_MODULE_X’
   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
   ^~~~~~~~~~~~~
../src/unix_dgram.cc:404:1: note: in expansion of macro ‘NODE_MODULE’
 NODE_MODULE(unix_dgram, Initialize)
 ^~~~~~~~~~~
In file included from /cache/node-gyp/10.24.0/include/node/node.h:63,
                 from ../../nan/nan.h:56,
                 from ../src/unix_dgram.cc:5:
/cache/node-gyp/10.24.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]’:
/cache/node-gyp/10.24.0/include/node/node_object_wrap.h:84:78:   required from here
/cache/node-gyp/10.24.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<node::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/cache/node-gyp/10.24.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]’:
../../nan/nan_object_wrap.h:65:61:   required from here
/cache/node-gyp/10.24.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
  SOLINK_MODULE(target) Release/obj.target/unix_dgram.node
  COPY Release/unix_dgram.node
make: Leaving directory '/src/repo/node_modules/unix-dgram/build'

> heapdump@0.3.15 install /src/repo/node_modules/heapdump
> node-gyp rebuild

make: Entering directory '/src/repo/node_modules/heapdump/build'
  CXX(target) Release/obj.target/addon/src/heapdump.o
In file included from ../src/heapdump.cc:17:
../../nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../../nan/nan.h:2294:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from ../src/heapdump.cc:15:
../src/heapdump.cc: At global scope:
/cache/node-gyp/10.24.0/include/node/node.h:573:43: warning: cast between incompatible function types from ‘void (*)(Nan::ADDON_REGISTER_FUNCTION_ARGS_TYPE)’ {aka ‘void (*)(v8::Local<v8::Object>)’} to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
       (node::addon_register_func) (regfunc),                          \
                                           ^
/cache/node-gyp/10.24.0/include/node/node.h:607:3: note: in expansion of macro ‘NODE_MODULE_X’
   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
   ^~~~~~~~~~~~~
../src/heapdump.cc:136:1: note: in expansion of macro ‘NODE_MODULE’
 NODE_MODULE(addon, Initialize)
 ^~~~~~~~~~~
In file included from /cache/node-gyp/10.24.0/include/node/node.h:63,
                 from ../src/heapdump.cc:15:
/cache/node-gyp/10.24.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]’:
/cache/node-gyp/10.24.0/include/node/node_object_wrap.h:84:78:   required from here
/cache/node-gyp/10.24.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<node::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/cache/node-gyp/10.24.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]’:
../../nan/nan_object_wrap.h:65:61:   required from here
/cache/node-gyp/10.24.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
  SOLINK_MODULE(target) Release/obj.target/addon.node
  COPY Release/addon.node
make: Leaving directory '/src/repo/node_modules/heapdump/build'

> gc-stats@1.4.0 install /src/repo/node_modules/gc-stats
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
[gc-stats] Success: "/src/repo/node_modules/gc-stats/build/gcstats/v1.4.0/Release/node-v64-linux-x64/gcstats.node" is installed via remote

> core-js@3.9.1 postinstall /src/repo/node_modules/core-js
> node -e "try{require('./postinstall')}catch(e){}"

Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!

The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: 
> https://opencollective.com/core-js 
> https://www.patreon.com/zloirock 

Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)


> nodemon@2.0.7 postinstall /src/repo/node_modules/nodemon
> node bin/postinstall || exit 0

Love nodemon? You can now support the project via the open collective:
 > https://opencollective.com/nodemon/donate

added 865 packages in 20.045s

$ npm test

> cxserver@1.2.0 test /src/repo
> npm run lint && npm run unittest


> cxserver@1.2.0 lint /src/repo
> eslint .


/src/repo/lib/dictionary/jsondict/JsonDict.js
  12:1  warning  Found more than one @return declaration  jsdoc/require-returns
  12:1  warning  Found more than one @return declaration  jsdoc/require-returns-check

/src/repo/lib/lineardoc/TextBlock.js
  34:2  warning  Found more than one @return declaration  jsdoc/require-returns
  34:2  warning  Found more than one @return declaration  jsdoc/require-returns-check

/src/repo/lib/mt/MTClient.js
  311:2  warning  Found more than one @return declaration  jsdoc/require-returns
  311:2  warning  Found more than one @return declaration  jsdoc/require-returns-check
  368:2  warning  Found more than one @return declaration  jsdoc/require-returns
  368:2  warning  Found more than one @return declaration  jsdoc/require-returns-check
  422:2  warning  Found more than one @return declaration  jsdoc/require-returns
  422:2  warning  Found more than one @return declaration  jsdoc/require-returns-check

✖ 10 problems (0 errors, 10 warnings)


> cxserver@1.2.0 unittest /src/repo
> PREQ_CONNECT_TIMEOUT=15 mocha



  Adaptation tests
    ✓ Link adaptation basic test (361ms)
    ✓ Link adaptation - Full paragraph and relative links (38ms)
    ✓ Image adaptation basic test (104ms)
    ✓ Image adaptation - Inline image with figure-inline tag
    ✓ Image adaptation - Inline image with figure-inline tag
    ✓ Image adaptation - Source and target language directions differs (126ms)
    ✓ Image adaptation - Image is not in commons
    ✓ Gallery adaptation (1164ms)
    ✓ Reference adaptation test without a reference template
    ✓ Reference adaptation test where the content is reference and template at same time
    ✓ Reference adaptation test where source and target has no template data. Must use template mapping database (105ms)

  Adaptation tests
    ✓ should adapt section when: section has lot of templates, but all are fragments of main template (310ms)

  Template parameter mapping test
    ✓ should not have any errors while mapping params

  JsonDict
    ✓ should return result when looking up for an existing word
    ✓ should return no result when looking up for an non existing word

  express app
starting test server
    ✓ should get robots.txt
    ✓ should set CORS headers
    ✓ should set CSP headers

  Swagger spec
    ✓ get the spec
    ✓ spec validation
    routes
      ✓ robots.txt check
      ✓ root with no query params
      ✓ spec from root
      ✓ root with wrong query param
      ✓ Fetch enwiki protected page (176ms)
      ✓ Fetch dictionary meaning with a given provider
      ✓ Fetch dictionary meaning without specifying a provider (39ms)
      ✓ Machine translate an HTML fragment using TestClient. (62ms)
      ✓ Get the tools between two language pairs
      ✓ Get all the language pairs
      ✓ Get the MT tool between two language pairs
      ✓ Translate enwiki protected page (157ms)
      ✓ Machine translate an HTML fragment using TestClient, adapt the links to target language wiki. (231ms)
      ✓ Suggest a source title to use for translation (268ms)
      ✓ Suggest target section titles for given source sections
      ✓ Suggest source sections to translate (172ms)
      ✓ retrieve service info
      ✓ retrieve service name
      ✓ retrieve service version
      ✓ redirect to the home page

  service information
    ✓ should get the service name
    ✓ should get the service version
    ✓ should redirect to the service home page
    ✓ should get the service info

  v1 api - page gets
    ✓ should get the whole page body (573ms)
    ✓ should throw a 404 for a non-existent page

  v2 api - page gets
    ✓ should get the whole page body (544ms)
    ✓ should throw a 404 for a non-existent page (105ms)

  LinearDoc
    ✓ should be possible to linearise all kind of HTML inputs (45ms)
    ✓ should be possible to reconstruct the HTML from LinearDoc
    ✓ should be possible to reduce and expand a document
    ✓ test HTML compaction roundtrip with inline chunks
    ✓ test HTML expand with external attributes inserted
    ✓ test if the content is block level template (165ms)
    ✓ test HTML compaction roundtrip with inline style content
    ✓ test HTML compaction roundtrip with template with empty content
    ✓ test getRootItem for ignoring blockspaces
    ✓ test getRootItem for not ignoring non-whitespace content in textblock
    ✓ test getRootItem for ignoring whitespace content in textblock

  Apertium machine translation
    ✓ Test: All caps words
    ✓ Test: Title caps one-to-many
    ✓ Test: Reordering with nested tags
    ✓ Test: Many-to-one with nested tags
    ✓ Test: Reordering at either ends of a tag
    ✓ Test: Identical tags separated by whitespace
    ✓ Test: Non-identical links separated by whitespace
    ✓ Test: Find longest match among multiple matches
    ✓ Test: Do not translate content inside the style tags

  Google machine translation
    ✓ Should fail because of wrong key 

  LingoCloud machine translation
    ✓ Should fail because of wrong key 

  Machine translation with wrapped html result
    ✓ Should throw error

  Machine translation result with extra spaces
    ✓ Should be cleaned up

  Subsequence match finding
    ✓ Should return correct range mapping

  Matxin machine translation
    ✓ Should fail because of wrong key 

  Template translation
    ✓ should not translate the fragement contents.

  Yandex machine translation with corrupted result
    ✓ Should sanitize the MT output

  Yandex machine translation
    ✓ Should fail because of wrong key 

  Youdao machine translation
    ✓ Should fail because of wrong key 

  MWPageLoader tests
    ✓ Test: Add data-section-number attribute to every CX section

  Section wrap tests
    ✓ should parse correctly when section has common pattern of elements
    ✓ should extract correct number of categories when section has common pattern of elements
    ✓ should parse correctly when section has categories to be extracted
    ✓ should extract correct number of categories when section has categories to be extracted
    ✓ should parse correctly when content has nested sections and tricky transclusion context
    ✓ should extract correct number of categories when content has nested sections and tricky transclusion context
    ✓ should parse correctly when content has blank template and then an unrelated table
    ✓ should extract correct number of categories when content has blank template and then an unrelated table
    ✓ should parse correctly when content is complete page content with html, head tags and body having two templates with fragments
    ✓ should extract correct number of categories when content is complete page content with html, head tags and body having two templates with fragments
    ✓ should parse correctly when Content has self closing meta tag
    ✓ should extract correct number of categories when Content has self closing meta tag
    ✓ should parse correctly when Content has template fragments and one fragment is a section candidate. Section has categories too
    ✓ should extract correct number of categories when Content has template fragments and one fragment is a section candidate. Section has categories too
    ✓ should parse correctly when Content has transclusion and one of its fragment get removed since it is ignorable.
    ✓ should extract correct number of categories when Content has transclusion and one of its fragment get removed since it is ignorable.
    ✓ should parse correctly when Content has transclusion and same element is removable templatestyle. So do not remove
    ✓ should extract correct number of categories when Content has transclusion and same element is removable templatestyle. So do not remove

  Title pair tests
    ✓ should adapt the title when: Corresponding title exist in target language
    ✓ should adapt the title when: Corresponding title exist in target language
    ✓ should adapt the title when: Corresponding title exist in target language and given title need normalization
    ✓ should adapt the title when: Corresponding title does not exist in target language and given title need normalization
    ✓ should adapt the title when: Corresponding title exist in target language and given title need normalization, has parenthesis
    ✓ should adapt the title when: Source title is percentage encoded, but should not throw error, should return corresponding title

  Title pair tests - batching
    ✓ should have the queue size 50

  Segmentation tests for am
    ✓ should not have any errors when: Amharic segmentation - basic test

  Segmentation tests for en
    ✓ should not have any errors when: Simple paragraph test
    ✓ should not have any errors when: Sentence ending with )
    ✓ should not have any errors when: Exclamation, punctuation test
    ✓ should not have any errors when: Multiple sentences
    ✓ should not have any errors when: Multiple sections - div, h1, paragraph
    ✓ should not have any errors when: Sentence starting with a link
    ✓ should not have any errors when: Sentence starting with a link and ending with a link
    ✓ should not have any errors when: Figure
    ✓ should not have any errors when: References can appear after period without space. Example: Hydrogen is a gas.[1] It is ...
    ✓ should not have any errors when: References can appear after period without space, repeated. Example: Hydrogen is a gas.[1][2][3] It is ...
    ✓ should not have any errors when: References can appear inside a sentence. Example: Hydrogen is a gas[1] and it is ...
    ✓ should not have any errors when: References can appear inside a sentence. Example:Hydrogen's atomic number is one. Hydrogen is a gas[1] and it is ..... Test the flags are reset properly
    ✓ should not have any errors when: References can appear repeated inside a sentence. Example: Hydrogen is a gas[1][2][3] and it is ...
    ✓ should not have any errors when: References can appear after period and space. Example: Hydrogen is a gas. [1] It is ...
    ✓ should not have any errors when: Paragraph from Debian article- repeating references
    ✓ should not have any errors when: References after closing bracket and period.
    ✓ should not have any errors when: Three sentences in a single text node
    ✓ should not have any errors when: Paragraph ending with reference and already having reference in between. The second reference should not be identified as repeating reference. If identified as repeating reference, 7 letters from last word will be missing.
    ✓ should not have any errors when: Sentences ending with abbreviations
    ✓ should not have any errors when: link element inside span element
    ✓ should not have any errors when: Do not break inside of links with abbreviations. Example: Carl E. Wieman. See https://phabricator.wikimedia.org/T99504
    ✓ should not have any errors when: A non-segmentable text chunk inside a textblock should not mark the whole text chunk non-segmentable
    ✓ should not have any errors when: Figure-inline segmentation. Do not violate the MediaWiki DOM spec https://www.mediawiki.org/wiki/Specs/HTML#Images
    ✓ should not have any errors when: Figure-inline inside a transclusions
    ✓ should not have any errors when: Video tag in figure tag
    ✓ should not have any errors when: Audio tag in figure tag
    ✓ should not have any errors when: Audio tag in figure-inline tag
    ✓ should not have any errors when: Video tag in figure-inline tag
    ✓ should not have any errors when: Do not segment content inside block templates
    ✓ should not have any errors when: about attribute alone does not initiate a transclusion context
    ✓ should not have any errors when: Do not consider style tag with transclusion as block tag

  Segmentation tests for de
    ✓ should not have any errors when: Avoid segmenting inside a transclusion textblock

  Segmentation tests for hi
    ✓ should not have any errors when: Hindi segmentation - basic test

  Segmentation tests for hy
    ✓ should not have any errors when: Armenian segmentation - basic test

  Segmentation tests for ja
    ✓ should not have any errors when: Japanese segmentation - basic test

  Segmentation tests for zh
    ✓ should not have any errors when: Chinese segmentation - basic test

  Segmentation tests for pa
    ✓ should not have any errors when: Punjabi segmentation - basic test

  SectionSuggester tests
    ✓ should find present and missing sections

  Link Adaptation tests
    ✓ Basic test (151ms)
    ✓ Space in the title
    ✓ Relative URL
    ✓ Title with parenthesis
    ✓ Target title does not exist
    ✓ Title with translation from Wikidata label
    ✓ ISBN Link adaptation

  Reference adaptation
    ✓ Adapting a reference with plain text content
    ✓ Adapting a reference with plain text content, sup tag
    ✓ Adapting a reference with html content, sup tag
    ✓ Not adapting a reference with no data-mw.body
    ✓ Mark the reference adaptation failed when the content defined in template not adapted (211ms)
    ✓ Mark the reference adaptation failed when the content defined in template and it is unsupported by CXserver
    ✓ Partial reference adaptation, when mandatory params are not transferred successfully (121ms)

  Template adaptation
    ✓ Template with no equivalents (82ms)
    ✓ Babel template (100ms)
    ✓ Cite web template (178ms)
    ✓ Multi-part template for columns (80ms)
    ✓ Template with equivalents, but target template data missing, no params mapped (74ms)
    ✓ Equivalient template exist, templatedata exists only for source language. Use that as hint for extracting params from template source code in target language (117ms)
    ✓ Template with equivalents, but not able to map mandatory params (90ms)

stopping test server

  164 passing (7s)


$ git add .

$ git commit -F /tmp/tmpr0b2yrhn
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
Source code is licensed under the AGPL.