- poikilotherm (CEST,UTC+2)*KRoy Pardee (KaPoW): to extend a bit on this: making data FAIR does not require public availability of data. Metadata publications are pretty common and one of the usual use cases of Dataverse. It usually is called a "registry" in these cases instead fof "repository". In our installation, we try to focus on the metadata and keep the data out - mostly not because of paranoia but because our institutes very often do not get storing the data longterm wrong and we see not much use in duplicating the data. Also, we have a lot of data that is rather large (TB/PB) and that is simply impossible to have a copy of. In these cases, metadata publications make up a pretty good alternative, as the data is at least FAIR and referencable.
- *KRoy Pardee (KaPoW): to extend a bit on this: making data FAIR does not require public availability of data. Metadata publications are pretty common and one of the usual use cases of Dataverse. It usually is called a "registry" in these cases instead fof "repository". In our installation, we try to focus on the metadata and keep the data out - mostly not because of paranoia but because our institutes very often do not get storing the data longterm wrong and we see not much use in duplicating the data. Also, we have a lot of data that is rather large (TB/PB) and that is simply impossible to have a copy of. In these cases, metadata publications make up a pretty good alternative, as the data is at least FAIR and referencable.
- (We actually did take this one step further and added a new field type to allow real URLs. This is a fork... https://data.fz-juelich.de/guide/juelich/data-linking.html)
- prsridhaP
- Thanks for looking into it! This is how it's looking from my end. I have deleted the files manually for now.
- pdurbinPprsridha: manually? In smaller batches? One at a time? Using the API? Sorry, I'm not sure what you mean. 😅 Thanks for the screenshot! Is that from the demo server? Or your own installation?
- Pallinger PéterPHello! One of our test servers fails to run the dataverse installer (I tried to install v5.13), due to
"Internal Exception: java.sql.SQLException: Error in allocating a connection. Cause: Access denied to execute this method : setURL" - I am a bit stumped to how to debug this issue. I could just reformat the whole VM, but I would like to know why this problem occurs...
- It would seem that the sql password is bad, but it is the same as on out other test server, which is working fine...
- First, I would like to know how the POSTGRES_PASSWORD in the default.config is used... How can I check whether it is set correctly?
- * First, I would like to know how the POSTGRES_PASSWORD in the default.config is used... How can I check whether it is set correctly in payara?
- Pallinger PéterPOr could the cause possibly be that setURL is an sql function that is not defined in the current database?
- pdurbin
I only see
setURL
once in the codebase and it's not a SQL thing:$ ack setURL src src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java 1609: public String getReturnToDatasetURL(String serverName, Dataset dset) {
- Pallinger PéterPI think it is in eclipselink, org.eclipse.persistence. ....
- * I think it is in eclipselink, org.eclipse.persistence. .... I dug somewhat into the source code after examining the stack trace...
- I would seem that I somehow misset the SQL connection parameters, but I am still not sure what parameters the server uses. The postgres server itself does not report any connection attempts in the log, although I dialed up the log level to DEBUG1...
- Which would point to the server adress to be misset, but it seems correct in the domain.xml (is that the correct place for sql connection params like dataverse.db.host, not?).
- * Which would point to the server adress to be misset, but it seems correct in the domain.xml (is that the correct place for sql connection params like dataverse.db.host?).
- I can manually connect to postgres with psql, giving the apparently same parameters that dataverse is set to use...
- pdurbinOk. So you're saying if I use the wrong password for postgres I should expect this error?
- Pallinger PéterPmaybe, I will try to set the wrong password on my other test server...
- Pallinger PéterPUnfortunately, the password error is different:
FATAL: password authentication failed
for user "dvnuser" - so, I am again stumped about the cause of the above setURL error...
- pdurbinAny ideas,Opoikilotherm (CEST,UTC+2) ?
- poikilotherm (CEST,UTC+2)setURL is coming from the underlying JDBC class and refers to some wrong JDBC URL
- Please check your credentials
- AFAIK the installer is battle tested to create the aliases etc
- But of course something might have gone wrong there
- pdurbincontainerization working group meeting in 5 minutes: https://ct.gdcc.io
- Miguel SilvaAs soon I finish prototyping my smart DAQ with swarm capabilities I intend to take a peak at the dataverse source code to propose some improvements in regards to live data acquisition and open science ... Almost there
In reply to
pdurbincontainerization working group meeting in 5 minutes: https://ct.gdcc.io - Here's the newest smart DAQ with SWARM capabilities
- Available on my public GitHub profile to anyone use
- ( Open hardware with CC licensing )
- @archive:matrix.org joined the room
- pdurbinNew logs for this channel! Check 'em out! https://archive.matrix.org/r/dataverse:matrix.org
- pdurbinI just switched us over: https://github.com/IQSS/chat.dataverse.org/pull/24
- poikilotherm (CEST,UTC+2)Fancy pants!
- pdurbinI know right
- pdurbinMy daughter is stage crew for her 8th grade musical, Finding Nemo Jr., premiering two hours from now! I'm off! Have a great weekend! 🐟️
- imlostlmao joined the room
- tosigus joined the room
- Don SizemoreD@poikilotherm
dataverse.files.directory
is only used for temporary files these days, correct? I'm asking because I ran acrosstarget/classes/META-INF/microprofile-config.properties:dataverse.files.directory=/tmp/dataverse
- poikilotherm (CEST,UTC+2)No it's not!
- There is a list in the docs where this stuff is used
- (I asked myself the same thing when going for Mpconfig)
- This is a sane default for most installations and development
- Btw this directory is the default which had been present in the code before as well
- pdurbina pre-existing condition 🤒
- Don SizemoreD@poikilotherm if u hapi i'm hapi, just double-checking
- pdurbinAwesome thread about running Dataverse in containers for API testing: https://dataverse.zulipchat.com/#narrow/stream/377090-python/topic/containers.20for.20API.20testing
- pdurbinCommunity call in a little under 3 hours, including a demo of Pooch! https://dataverse.org/community-calls
- pdurbinPooch is now on DataverseTV: https://groups.google.com/g/dataverse-community/c/GKJty_qNDbU/m/BtTnOrmCAwAJ
- zbentaZPooch looks promissing, it would be great if we could download all the files, metadata and permissions associated with a said dataset and upload them to a different dataverse instance.
- zbentaZWe asked our CERT team to do a blackbox security check of our dataverse installation, they found two vulnerabilities, one regarding the jQuery UI 1.13.0 version (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-31160) and another regarding the absence of Anti-CSRF tokens. Should we create an issuePpdurbin ? The version of dataverse we are using is v5.13.
- pdurbinZzbenta: yuck. Can you please email our security address? https://guides.dataverse.org/en/5.13/installation/config.html#reporting-security-issues
- Zzbenta: as for moving datasets from one installation to another, I believe this is now possible using the BagIt format.
- pdurbin
This message is being deleted (typo)…
- Zzbenta: for more on round trip and BagIt, please see https://groups.google.com/g/dataverse-community/c/07h0Ca-Ai1I/m/19U7Xlt7AwAJ
- zbentaZZzbenta: yuck. Can you please email our security address? https://guides.dataverse.org/en/5.13/installation/config.html#reporting-security-issues
I'll do so.
- pdurbinThanks!
- zbentaZDataverse also has an issue with the session cookie being sent in cleartext, we tried using a session cookie from a coleague in my browser and I was able to impersonate him.
- pdurbinZzbenta: please email security about that too
- Zzbenta: also, this channel is logged so it's not the best place to talk about security 😅
- zbentaZ
Dataverse also has an issue with the session cookie being sent in cleartext, we tried using a session cookie from a coleague in my browser and I was able to impersonate him.
You can skip this one, it was a misunderstanding, as fot the others, we have sent an email regarding the issues.
- *
Dataverse also has an issue with the session cookie being sent in cleartext, we tried using a session cookie from a coleague in my browser and I was able to impersonate him.
You can skip this one, it was a misunderstanding, as for the others, we have sent an email regarding the issues.
- pdurbinThanks!
- Zzbenta: I can also give you access to https://github.com/IQSS/dataverse-security/issues if you like. Please just let me know.
- zbentaZSure thing, why not
- pdurbinZzbenta: done! Do you see an invite at https://github.com/IQSS ?
- Santiago Florez joined the room
- Santiago FlorezSHello everyone
I want to change the way to display the metadata citation in the subject field. I need the field to be displayed in URL format. I've been thinking about editing the tsv. - pdurbinSSantiago Florez: if you get it working, please open a pull request! ❤️
- Do you have a screenshot or a mockup of how you would like it to look?
- Santiago FlorezSYes, hold on a moment
- pdurbinInteresting. And where does "Chemistry" link to?
- Santiago FlorezSwhen I click on it, I search for that metadata in all Dataverses and datasets.
- pdurbinI see
- Santiago FlorezSi´ve modified the displayFormat in the tsv to "<a href='/dataverse/#{dataverse.alias}?q=""#VALUE""' target=""_blank"" rel=""noopener"">#VALUE</a>"
- pdurbinAnd maybe the same for author. If you click an author, you see all the datasets by that author?
- Santiago FlorezSNoup, I just need to change that field
- And I've tried running solr's update***.sh. I don't know if I need to do something else.
- pdurbindid you reload citation.tsv? with curl?
- Santiago FlorezSjmmm no
- I gonna try
- Santiago FlorezSOnce I run the curl command, should I see changes to a new data set? or on all the datasets that I had created
- pdurbinboth