WEBVTT 00:00.000 --> 00:10.800 Tolling and how to use them, and that might be overwhelming for users and users and people 00:10.800 --> 00:16.640 new to that stuff, Ryan will talk about how that might be made easier for the community. 00:16.640 --> 00:17.640 Thank you. 00:17.640 --> 00:20.360 Yeah, there's been a lot of adfishing, it feels like doing the presentation in the morning 00:20.360 --> 00:23.640 is the right one to do, but there's the recordings I suppose as well. 00:23.640 --> 00:29.920 So you've heard a lot about the variety of tools today, I think there was. 00:30.320 --> 00:37.760 Acid update, EOS, there was, they won by TNO, SN, as pipes, there was very good, there's 00:37.760 --> 00:42.160 variety of tools that you've heard today, and maybe about five or six different tools, 00:42.160 --> 00:49.120 and they probably constitute on the order of about 200 open-source energy system modeling tools, 00:49.120 --> 00:50.800 being really, really specific here. 00:50.800 --> 00:56.160 These are tools for infrastructure planning and operation, open-source, that we know about. 00:56.720 --> 01:02.720 And it means that it's a real big problem to keep track of the tools that are available, 01:02.720 --> 01:05.760 unless you come to every single possible conference for people to talk about these things, 01:05.760 --> 01:10.880 and maybe you'll pick up one or two each time, or you read review papers, 01:10.880 --> 01:15.360 because actually things often come from the academic literature, or academic community, 01:15.360 --> 01:17.600 these things often developed in academia. 01:17.600 --> 01:23.280 And so you could go back, you could read a review, pretty much every single year, 01:23.280 --> 01:28.320 there will be a review about the latest developments in open-source, or just general energy 01:28.320 --> 01:34.400 system modeling, maybe even in 2022, there was about five or six reviews that came out 01:34.400 --> 01:41.280 in the same journal, that what are the latest developments in energy system modeling and the 01:41.280 --> 01:47.600 tools available, and it's effectively a losing battle, because you read these, you might get 01:47.600 --> 01:52.160 the latest developments in the tools that they know are available, they might grab about 100 01:52.160 --> 01:58.400 maximum of the 200 of there, and they are also probably about two or three years old by the time 01:58.400 --> 02:02.320 you read this paper, right? So they've started their review as a PhD student, probably, 02:02.880 --> 02:07.840 they've taken some time doing the review, they actually literally, the academic review process, 02:07.840 --> 02:12.160 the peer review process, and then you read it a few years later, and inevitably they got a lot of 02:12.160 --> 02:17.440 things wrong. So I've been a lead developer of one of these 200 tools for about nine years, 02:18.160 --> 02:22.240 and I can tell you, every time I read one of these, they've always got something wrong 02:22.240 --> 02:27.440 about what my tool can do, because they are reading it from an external perspective, 02:27.440 --> 02:32.880 and maybe that's a sign of me or my documentation, but it's true for a lot of the other tools 02:32.880 --> 02:38.800 as well that I know about. So this is where it comes in a project that we've been undertaking at the 02:38.800 --> 02:44.720 open-energy transition with support from breakthrough energy, grid initiative, to map all of these 02:44.720 --> 02:51.600 tools that are available in a fashion that will stay up to date continuously. So what you see here 02:51.600 --> 02:56.800 is a snapshot of the latest update, it's a monthly update we get, so this is the January update 02:56.800 --> 03:05.440 in terms of a snapshot of the top sort of 15 or so tools of these 210 that we've got in the inventory. 03:06.640 --> 03:12.720 I can show you, I hope, what that looks like in a bit more detail here, so this is, 03:12.720 --> 03:18.800 this is it live, openmod-tracker.org, and we've got a couple of caveats at the start, 03:18.800 --> 03:22.560 so I want to be very clear, we believe we're catching most of the open source tools that are 03:22.560 --> 03:27.840 out there, we're probably aren't catching them all, and don't blame us for the data associated 03:27.840 --> 03:32.800 with your tool, right? Well, I'll tell you in a minute, we get most of this data from elsewhere. 03:33.600 --> 03:40.400 So you've got the table here, a couple of different pieces of information that might be useful 03:40.400 --> 03:43.760 to you when you come and look at it, you've got the link to the source code, so all of these 03:43.760 --> 03:51.280 have to have openly available, publicly available source code, you can get a link to the documentation, 03:51.280 --> 03:58.240 and we have a link to all the 60 or so tools documentation, so this is a really good resource, 03:58.240 --> 04:02.720 if you just want to find out how to use a tool, I think we do a better job than the 04:02.720 --> 04:09.040 readmees of most of the repositories of telling you where their documentation is, so check that out, 04:09.040 --> 04:13.920 and then a variety of metrics associated with the source code repositories for these tools, 04:13.920 --> 04:19.040 so things like when was it created, when was it most recently updated, how many stars does it have? 04:19.040 --> 04:26.000 This is ordered by stars at the moment. Number of contributors and the development distribution 04:26.000 --> 04:31.600 score that I believe was developed in open sustained tech, and both of those really are sort of a 04:31.600 --> 04:36.080 bus factor, it's trying to tell you, you know, if there's one or two developers and a really, 04:36.080 --> 04:42.000 really low DDS, it's telling you that if they go out of action, they choose to leave the industry, 04:42.000 --> 04:46.400 that entire project will probably die with them, so you want lots of contributors, you want a 04:46.400 --> 04:50.240 high DDS, you want a lot of force, you want people interacting with it in lots of different ways, 04:51.040 --> 04:55.200 and you want to know how often it's downloaded, and that's a difficult one because we can only 04:55.200 --> 05:00.480 get that for verifiable downloads, things that are indexed on package indexes, usually price and 05:00.480 --> 05:05.840 packages, is what that's possible for, and a couple of other data sets as well. You can do things 05:05.840 --> 05:10.240 like filter, you can say, well, I only want to see ones that have documentation, maybe ones that 05:10.240 --> 05:15.440 are at least, you know, five or six years old, I want to have a bit of pedigree, maybe you want 05:15.440 --> 05:20.320 ones that have a certain number of stars associated with them, so you can do a variety of filtering 05:20.320 --> 05:24.960 actions to take you through and really filter down on the tools that might be of interest to you, 05:24.960 --> 05:29.520 for whatever project you might be doing, whether that's as a user, potential user for a tool, 05:29.520 --> 05:35.440 or potential developers, and be joining the development community for a tool, there's a lot of possibilities 05:35.440 --> 05:40.960 there. You can also deep dive into a few other things, so we've tried to capture all of the 05:40.960 --> 05:47.680 users of these tools across GitHub and GitLab, if they're on those ones, and try to classify them, 05:47.680 --> 05:51.680 to get an impression of the types of users of these tools, the people are interacting with 05:51.680 --> 05:56.480 these repositories. We don't do that quite a job, right? So about 65% of the users on GitHub 05:56.480 --> 06:01.840 who interacted with these tools, of the 50,000 or so that we're picking up, that's 16,000, and we 06:01.840 --> 06:07.280 don't get about 65% of them. That's because it's just that GitHub username and no other 06:07.280 --> 06:11.680 information, and we can't hate to do any better than that. But of those who do interact, we get a 06:11.680 --> 06:15.280 lot of people from the academic community, there's a little small few if you go through the actual 06:16.720 --> 06:20.720 thing you can have a look at any more detail, academic community, research organizations, 06:20.720 --> 06:26.000 professionals, and industry, they're all interacting in various ways. We can look at which 06:26.080 --> 06:33.840 organizations they are, NREL now, the national lab of the Rockies I think is a big one. We 06:33.840 --> 06:39.520 open energy transition and also a big one, a variety of other ones, so you've got things like 06:39.520 --> 06:46.000 Chinese universities or US universities. You can look at this on a two levels, that was for all tools, 06:46.000 --> 06:50.640 I'm just going to look at one of them here, maybe very good isn't here, that would get a very good. 06:50.720 --> 06:56.560 So this is really picking on you, but you come to mind, and we can see various organizations who 06:56.560 --> 07:01.360 got involved, there's countries where the users get involved, so you can see the international 07:01.360 --> 07:05.200 sort of community that might be involved with it. You can maybe look at another one on side, 07:05.200 --> 07:11.200 we have pipes afterwards, so we can add that in, we can see across those two tools, 07:11.200 --> 07:14.800 sort of the global reach in terms of the kinds of users are getting involved, at least the ones 07:14.800 --> 07:19.920 we know about, about 35%. And we can look at the project development metrics, so we can say 07:20.240 --> 07:25.200 for these projects, some information about their development cadence, right? How are they 07:25.200 --> 07:29.360 are they waxing? Are they waning in terms of their level of development? Poor requests, 07:29.360 --> 07:33.680 issues being opened, who are the top contributors across these? This is the top 07:33.680 --> 07:39.520 contributions across all of the projects, who are the top organizations, and in what way do they 07:39.520 --> 07:44.240 contribute? Do they contribute in terms of opening issues? Do they contribute in terms of reviewing 07:44.240 --> 07:51.760 or giving feedback? And what is the type of feedback we're getting? So times closed issues, 07:51.760 --> 07:56.960 time to merge, pull requests, all useful information for understanding how well a project is running, 07:56.960 --> 08:00.960 and how much engagement is happening during resolution? Is there good feedback on reviews? 08:00.960 --> 08:04.800 Can you trust the review process? Is there enough feedback on it from that perspective? 08:05.360 --> 08:10.080 So all of this is possible from this dashboard that we've created in order to help people 08:10.080 --> 08:15.440 understand the future of energy system modeling or where they might want to go? 08:17.680 --> 08:24.320 So what we do is we have to kind of herd cats to achieve this. We go to a variety of upstream 08:24.320 --> 08:31.440 inventories of these kinds of tools, and we grab them all in order to pull together all of the tools 08:31.440 --> 08:36.640 that we bring. So we know of these, if you know of more, really happy to know about them, 08:37.600 --> 08:41.920 we want to pull in as many as we can, and we do some triage kind of, we remove duplicates, 08:41.920 --> 08:47.440 we filter things out, exclude some tools that we just know for sure, repositories that we know 08:47.440 --> 08:51.360 for sure are not energy system modeling tools, but they've made their way through to these upstream 08:51.360 --> 08:59.200 inventories, and they also have to have open and public source code repositories and public 08:59.280 --> 09:08.080 licenses, like permissible, permissible copy left licenses usually. We tend to find that actually 09:08.080 --> 09:13.360 our just getting stuff in academic reviews and building a manual data set as the best of all of them 09:13.360 --> 09:18.160 in terms of unique tools that you don't find anywhere else, and some inventories don't provide 09:18.160 --> 09:22.720 any new tools to the mix. So they have tools that they've mentioned elsewhere as well, so we don't 09:22.720 --> 09:29.120 actually add any new information for us. We bring this together with the ecosystem's API 09:29.120 --> 09:34.560 and I'm sure Andrew and this bit, I think it is, well, it's probably somewhere at first end this 09:34.560 --> 09:39.200 weekend and you should really find them if you can, because ecosystems are great API to take you 09:39.200 --> 09:43.360 to make you something with your all of our metrics, and now you can also blame them when the 09:43.360 --> 09:47.840 information looks weird, because we aren't really, we're not doctoring that information, we take it 09:47.840 --> 09:54.400 and we just show it as it's. We also do some scraping to try and find the dots pages in a bit more 09:54.400 --> 09:59.600 detail, because we don't get that well from elsewhere. That gives us some stats, and then we go 09:59.600 --> 10:04.880 directly to some DRA APIs for GitHub and GitHub, and that gives us the information about our 10:04.880 --> 10:11.920 users and about the repositories interactions over time. This is all run in about an hour and 10:11.920 --> 10:19.120 20 minutes in a GitHub CI. The time is mostly in this GitHub and GitHub API requests, but we run 10:19.120 --> 10:23.680 that just automatically, it's sort of set and forget now that it should happen monthly. I say that 10:23.680 --> 10:28.240 but the last one failed and I had to fix some bugs, but in theory it should run set and forget. 10:28.240 --> 10:34.400 So instead of you having to wait for the next review next year on the tools that are available 10:34.400 --> 10:41.600 that comes out of academia, you can check this dashboard instead. So why might you use it? 10:41.600 --> 10:46.080 Like who's going to it? And we at the Open Energy Transition, we've got sort of a mission 10:46.080 --> 10:53.120 a vision to make open source the norm in energy system planning by 2028. That means going to energy 10:53.120 --> 10:58.080 system operators, transmission system operators, and working with them to move them away from proprietary 10:58.080 --> 11:04.400 tools to using open source tools for their energy planning. That's our entire goal. And that means 11:04.400 --> 11:08.000 helping them understand what are the open source tools that are available in which one might be 11:08.000 --> 11:16.000 most appropriate to them for a given use case. And we do this by helping them with the tracker here, 11:16.400 --> 11:22.720 to produce a long list of possible tools for them. And then we deep dive. We let them deep dive 11:22.720 --> 11:29.120 if they want to into more information about those tools. You can't hope to look at 200 tools, 11:29.120 --> 11:36.640 but you might be able to look at 10 or so. And that helps you create a short list. So with one client 11:36.640 --> 11:41.200 recently, back in August, we started some things. We just had some really broad brush things. 11:41.200 --> 11:45.440 There has to be documentation. You know, it has to be easy for them. If we hand over the tool to 11:45.440 --> 11:49.600 the model to them, they have to have documentation. They can't rely on us. They have to rely on the 11:49.600 --> 11:56.160 tool itself. It should have an index package. The maintainers should care enough about it being 11:56.160 --> 12:01.760 easy to download cross platform. And usually that means having a downloadable index package and so 12:01.760 --> 12:07.760 some monthly download stats that we can grab. When you have to compile things and you are talking 12:08.000 --> 12:12.640 to somebody who's limited them from a technical knowledge as usually Excel on Windows, 12:13.680 --> 12:18.960 you have to work with that, not against it. And things like permissive source code licenses, 12:18.960 --> 12:22.720 so that if they internally have a tech team that wants to make some updates, they're not going 12:22.720 --> 12:28.800 to be limited by that. In this case, you also included copy left. We have one column that is 12:28.800 --> 12:34.560 manually prepared because we've got some information on the top 30 or so tools about what kind of 12:34.560 --> 12:38.080 modeling can you do? So you can get some really high-level information about the kind of 12:38.080 --> 12:46.080 energy model you can do and you can use that for filtering as well. I should say, so PIPSEC came up in 12:46.080 --> 12:50.320 the earlier conversation. They do rank the highest on a lot of things production cost modeling. 12:50.320 --> 12:54.080 When I mentioned that I'm a lead dev of the tool for the last nine years, it's not PIPSEC. 12:54.080 --> 12:57.840 So it's always quite galling to talk to clients and be like, actually, you know, the tool that's 12:57.840 --> 13:02.960 looking best here for you is PIPSEC. I kind of just have to say that with a straight face and 13:03.040 --> 13:08.720 and let it lie. There are other tools that do quite well in this specific filtering for this specific 13:08.720 --> 13:15.840 client and yes, like, they do well across these but can you choose what waiting to give them? 13:15.840 --> 13:20.240 Because some, you know, might matter a lot more to them than others and that, it's very difficult 13:20.240 --> 13:27.520 but waiting to forks versus DDS, let's say. So we went in and we did a much more detailed analysis 13:27.520 --> 13:31.280 and these are the kinds of things I'd love to automate in the dashboard and if you've got ideas 13:31.280 --> 13:36.880 of how to do it, I'd love to find out test coverage. I'd like to find a way to automate 13:36.880 --> 13:41.520 accessing test coverage information because people publish their test coverage or store it in different 13:41.520 --> 13:46.960 ways depending on the language that things written in and other decisions. The quality of their 13:46.960 --> 13:51.280 documentation. I might go to a documentation, it's a single page, this is how you install a good 13:51.280 --> 13:55.760 link. So it looks like they've got documentation but actually they don't, so that requires some manual 13:55.760 --> 14:01.520 stuff. Where can you get community supports? Where can you get data? We've had a lot of our energy 14:01.520 --> 14:05.360 modeling tools and the fact that you need reference data. If you can get a tool but you don't 14:05.360 --> 14:10.080 get any data with it. I need way to access the data and link it to that tool. That tool is functionally 14:10.080 --> 14:15.280 useless to a lot of energy system operators. So reference databases for tools, you know, other tools 14:15.280 --> 14:20.880 that can link into it really easily seamlessly is really important, etc. etc. So we can go through 14:20.880 --> 14:25.680 this process of shortlisting through this and we did this with them to shortlist to about 14:26.080 --> 14:30.880 three or four and that was really good and they found it really useful to have this sort of 14:32.160 --> 14:37.200 verifiable way to how do you shortlist tools so that they can then go back to their higher 14:37.200 --> 14:41.440 ups and say this is why we want to go open source and this is how we're going to do it and this is 14:41.440 --> 14:50.880 why we're choosing this tool. But we need to dig deeper because we just heard earlier about pipe 14:50.960 --> 14:56.800 so it's stochastic optimization. If you asked about pipes and you want to stochastic optimization 14:56.800 --> 15:00.960 six months ago the answer would have been it's not the tool for you but you wouldn't know that 15:00.960 --> 15:05.440 from any of these metrics, right? So we need to dig deeper into this and look at other features 15:06.160 --> 15:11.200 and we went with a proprietary tool feature set. I won't tell you which one because I don't want 15:11.200 --> 15:17.440 to get into legal ramifications of it and looked at what are the features that proprietary open 15:17.440 --> 15:22.960 proprietary energy system modeling tools have, themes them, don't have to remember all this now 15:22.960 --> 15:29.920 it's all available in the GitHub repository, grouped them and then created a variety of features 15:29.920 --> 15:36.560 so basically a feature taxonomy of energy system modeling tools but at one that isn't too technical 15:36.560 --> 15:41.760 it needs to be really understandable to potential users who as I said aren't super technical they just 15:41.760 --> 15:45.440 they know they want to achieve a certain task but they wouldn't necessarily know the technical 15:45.440 --> 15:53.360 words what that might be. That brought us on to a new dashboard dashboard you will probably tell 15:53.360 --> 15:56.800 if you go to any of these I'm not a front-end developer and actually I've seen a lot of stream 15:56.800 --> 16:01.120 that apps today in this room so I'm guessing that most of the stream are not front-end developers 16:01.120 --> 16:05.440 but it does the job and these are all prototypes so not in aim to be production ready 16:06.720 --> 16:11.840 and this is very similar to actually going to be an extension of the the other page 16:12.160 --> 16:17.440 and in which it's it's own page that you can go in you've got your tools here and then you've 16:17.440 --> 16:23.440 got specifically what are the features they've got grouped and most importantly and this is what 16:23.440 --> 16:28.960 you're not going to get that easily and any of these review papers verification of that feature existing 16:29.680 --> 16:35.360 all of these little superscript are a link to a page in their documentation or their source code 16:35.360 --> 16:40.560 their tests that proves that that particular feature exists but also help you understand how you 16:40.560 --> 16:46.400 activate that feature in that tool if you want it and whether something's in development rather than 16:46.400 --> 16:53.200 so this one here whether it's in development rather than actually ready. You can use it to also 16:53.200 --> 16:57.360 filter it based on the use case you've got and I'm not going to go into this detail because clearly 16:57.360 --> 17:03.920 I won't have enough time to do so but you could say okay I'm running an integrated resource plan 17:03.920 --> 17:09.600 which is usually what is done in the US been particular to make high-level decisions on the 17:09.600 --> 17:15.600 future asset capacities that you want and we've gone through and relatively with some degree of 17:15.600 --> 17:19.920 subjectivity we've gone through and said what are the features needed by this and this is a 17:19.920 --> 17:24.320 filter view of those features and then you can look at your tools and say okay which ones meet those 17:24.320 --> 17:31.520 features the best. You can do that side by side specifically on use cases so you could look at all of 17:31.520 --> 17:36.880 these and you could say okay look at it for pipes or how is pipes are doing for all of these 17:36.880 --> 17:42.560 next which other so you look for a single tool but you look across all the use cases and finally 17:42.560 --> 17:47.200 you could look and you could write your own use cases so you could say okay here's all of the options 17:47.200 --> 17:51.840 give you a little of a tool tapers to what we mean by it click all of the things you're interested in 17:52.880 --> 17:57.520 save it I'll just click a couple more save it and then you can go back 17:57.520 --> 18:11.520 to the use case comparison and I can look at it in that context or I can look at the tools 18:11.520 --> 18:17.360 comparison I can look at it in in that context as well so I can look at my customer use case 18:17.360 --> 18:23.200 and look at it in that context so it's a way for you to be able to and as I can see here you 18:23.360 --> 18:28.720 I can't scroll down so it's clearly a problem there but it's a way for you to be able to 18:29.840 --> 18:34.000 discuss with others what are the features we need for our tool and therefore what are the tools 18:34.000 --> 18:39.600 that are most appropriate this is a manual task right it requires the developers usually themselves 18:39.600 --> 18:43.840 to contribute to this so that's why you only see four tools and I'm hoping to see tomorrow very 18:43.840 --> 18:50.480 great on here right yeah so we'll be here shortly afterwards and then I'm sure that gems pie will be 18:50.560 --> 18:55.840 here soon too and the value of adding your tool to this is the fact that then it will be 18:55.840 --> 19:00.400 used in the decision making process by energy system operators because we will be showing them 19:00.400 --> 19:05.840 this dashboard right we won't be filtering it for pipes so that's it will be showing it to them 19:05.840 --> 19:14.960 vanilla so get it up there as quick as you can this is a little bit what it looks like inside the 19:14.960 --> 19:22.880 feature set so you've got just a simple animal config we we took sort of some inspiration from 19:22.880 --> 19:29.200 Conterforge as to feed stock maintainers so you've got maintainers of these lists they don't 19:29.200 --> 19:34.480 have to be the developers of the tool but you've got maintainers of the list and they will be 19:34.480 --> 19:40.320 always tagged if there's some need for them to do anything we've got version schema for all of these so 19:40.320 --> 19:45.520 you could have different features feature lists that are tied to different version schemas 19:46.960 --> 19:51.920 and this is where I said you've got these sort of source sources to the specific 19:51.920 --> 19:58.160 use cases and those will be checked in review and then similarly for use cases the the only 19:58.160 --> 20:02.880 addition is that we've got a list of the assumptions that go into that subjective use case list 20:03.840 --> 20:12.960 so we're going we want community engagement that's why I'm here today we want to improve 20:12.960 --> 20:17.040 UI and UX we'll probably get a front end developer to actually look at this and hopefully 20:17.040 --> 20:22.000 don't scream we want to add new insights we want to add things like what are the academic and 20:22.000 --> 20:28.240 ideally non academic uses for these tools we're going to rely quite a lot on ecosystems for that 20:28.240 --> 20:35.680 and open Alex we want longer term download stats in the last 12 months pie pie and anaconda 20:36.240 --> 20:41.920 and Julia there were 3 million downloads that weren't CI related downloads of all of these 200 20:41.920 --> 20:47.440 tools so last year 3 million downloads across all of those tools that is not capturing all of them 20:47.440 --> 20:52.800 because we're only capturing really Python so we're missing quite a lot to want to get more analysis 20:52.800 --> 20:57.680 on that and we're going to rely on ecosystems as well for security analyses of dependencies 20:58.320 --> 21:04.080 because energy system operators want to be certain about the security of these tools they're using 21:05.440 --> 21:09.520 we're going to be doing that variety of ways over the next sort of six months or so so really 21:09.520 --> 21:14.720 if you want to get involved get involved soon and looking at UI improvements we really want to 21:14.720 --> 21:19.600 refine this taxonomy make sure it's useful to everyone useful to developers so they understand 21:19.600 --> 21:25.840 what they're missing and useful to use it so they understand what they need and what they can 21:25.840 --> 21:32.480 get from the tools that they're using just as a sort of a case in point of how we use this so 21:32.480 --> 21:38.800 the this this sort of feature gap analysis that we've got here with the we've worked with the 21:38.800 --> 21:46.400 pipes to to look at all of these things that are missing created a roadmap on GitHub with issues 21:47.120 --> 21:53.760 which if all closed we'll close this gap between pipes and probably the leading proprietary tool 21:53.760 --> 21:58.640 for energy system modeling so you can go through this process by doing this yourself you can 21:58.640 --> 22:03.040 say well what am I missing words it's what proprietary tools can do and create a roadmap for 22:03.040 --> 22:13.360 yourselves to be able to close that great so that is me done these are the two dashboards but also 22:13.360 --> 22:18.400 the two GitHub repositories associated with them and as I say this was done with a lot of 22:18.480 --> 22:23.760 ultimately financial but also inspiration support from breakthrough energy that's which is this 22:23.760 --> 22:29.360 project thank you very much 22:32.960 --> 22:39.040 two minutes right rapid fire 22:42.240 --> 22:47.920 there will be tools that do low-medium voltage yes definitely but these are generic tools right 22:48.080 --> 22:54.400 so for the most part low-medium voltage is a data problem not a tool problem really so it's more 22:54.400 --> 22:59.760 about how the modeling is done in terms of the tools so yes there are several that can do low-medium 22:59.760 --> 23:09.440 voltage that was the question yes such a tip yeah do you know the you open source solutions 23:09.520 --> 23:14.160 catalogue and could there be some integration just to make sure that you're 23:14.160 --> 23:21.600 too hospitable and it ends up you can a lot so do I know that you open source solution 23:21.600 --> 23:28.080 catalog and could there be some synergies there I do know of it this is obviously global it's not 23:28.080 --> 23:34.560 just use specific in terms of its use cases but yeah we would want to link to it we should 23:34.560 --> 23:38.480 probably actually think about as an upstream inventory more than anything else so that we could 23:38.480 --> 23:42.480 potentially get more things from it that are index on it but not in the other ones we've got 23:44.080 --> 23:47.840 at the moment we're not thinking of them downstream what we've got to other other places 23:50.880 --> 23:54.160 how did you feel throughout the CIA really done 23:54.160 --> 23:59.520 pipeline has that automatically in their BigQuery database and that's the main one and a 23:59.520 --> 24:04.160 condo doesn't but it's far fewer downloads than pipeline and Julia does it automatically 24:05.040 --> 24:07.920 in terms of removing CI from downloads 24:18.160 --> 24:23.200 yeah would this help return on investment building open source yeah I mean what I'm hoping it does 24:23.200 --> 24:28.640 is stop anyone from building a new open source energy system modeling tool because that is probably 24:28.640 --> 24:33.360 the worst result on investment yeah it's already relevant we see that but that might be just 24:33.360 --> 24:37.440 because of where we're getting the data from that we're not capturing the latest the newest ones 24:38.240 --> 24:44.160 we are hoping that people will use this to support the financially support tools based on choosing 24:44.160 --> 24:48.000 tools that look like them and sure enough to be worth supporting and that have strong 24:48.000 --> 24:52.240 organizations backing them in terms of working on them so we're hoping that people will look at 24:52.240 --> 24:56.640 this also from a financial backing perspective as well not sure how they would work out return on 24:56.640 --> 25:00.880 investment yeah the key takeaway is that don't start developing a new tool there 25:03.360 --> 25:05.360 cool thank you